2024-04-23T01:35:27.489 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-23T01:35:27.489 DEBUG:teuthology.run:Teuthology command: teuthology --description crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/simple thrashosds-health workloads/radosbench-high-concurrency} --owner scheduled_yingxin@teuthology --verbose --name yingxin-2024-04-23_01:23:55-crimson-rados-ci-yingxin-crimson-fix-tri-mutex-4-distro-default-smithi --archive /home/teuthworker/archive/yingxin-2024-04-23_01:23:55-crimson-rados-ci-yingxin-crimson-fix-tri-mutex-4-distro-default-smithi/7669459 -- /home/teuthworker/archive/yingxin-2024-04-23_01:23:55-crimson-rados-ci-yingxin-crimson-fix-tri-mutex-4-distro-default-smithi/7669459/orig.config.yaml 2024-04-23T01:35:27.537 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-23T01:35:27.637 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yingxin-2024-04-23_01:23:55-crimson-rados-ci-yingxin-crimson-fix-tri-mutex-4-distro-default-smithi/7669459 branch: ci-yingxin-crimson-fix-tri-mutex-4 description: crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/simple thrashosds-health workloads/radosbench-high-concurrency} email: yingxin.cheng@intel.com first_in_suite: false job_id: '7669459' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yingxin-2024-04-23_01:23:55-crimson-rados-ci-yingxin-crimson-fix-tri-mutex-4-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: ci-yingxin-crimson-fix-tri-mutex-4 ceph: conf: client.0: debug ms: 1 debug objecter: 20 debug rados: 20 global: enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc osd_pool_default_min_size: 2 osd_pool_default_size: 3 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 osd pool default crimson: true paxos service trim min: 10 osd: bluestore block size: 96636764160 bluestore compression algorithm: snappy bluestore compression mode: aggressive bluestore fsck on mount: true bluestore rocksdb cf: false crimson alien thread cpu cores: 6-7 crimson osd obc lru size: 10 debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 err to stderr: true log flush on exit: true log to file: false log to stderr: true mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd debug reject backfill probability: 0.3 osd delete sleep: 1 osd failsafe full ratio: 0.95 osd max backfills: 3 osd max markdown count: 1000 osd objectstore: bluestore osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 2 osd.0: crimson seastar cpu cores: 0-2 osd.1: crimson seastar cpu cores: 3-5 osd.2: crimson seastar cpu cores: 0-2 osd.3: crimson seastar cpu cores: 3-5 flavor: crimson fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum sha1: a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: flavor: crimson sha1: a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: min_in: 4 workunit: branch: ci-yingxin-crimson-fix-tri-mutex-4 env: CRIMSON_COMPAT: '1' sha1: a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4 owner: scheduled_yingxin@teuthology priority: 60 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - osd.0 - osd.1 - client.0 - node-exporter.a - - mgr.x - osd.2 - osd.3 - client.1 - prometheus.a - node-exporter.b sha1: a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4 sleep_before_teardown: 0 suite: crimson-rados suite_branch: ci-yingxin-crimson-fix-tri-mutex-4 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4 targets: smithi090.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAWC5iJNafd1R+E2MWb4IxaYKCxUJ+HluqBuYCuGUXLeh2aLSeliTX1hcdEivTGrHmcBSK7uu5wXYfheVEoQH4I= smithi113.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCKdEn9EAfV/3De7+BKt/+rJPlg1u5r1QRzsikh2mPtC8ERZ/gQkM/mGeLMZrXk30QriId993Gxm6p+dUkuqAB4= tasks: - install: null - ceph: conf: osd: debug monc: 20 flavor: crimson - thrashosds: ceph_objectstore_tool: false chance_down: 0 chance_inject_pause_short: 0 chance_reset_purged_snaps_last: 0 chance_thrash_cluster_full: 0 chance_thrash_pg_upmap: 0 dump_ops_enable: false min_in: 3 noscrub_toggle_delay: 0 reweight_osd: 0 sighup_delay: 0 thrash_primary_affinity: false timeout: 2400 - full_sequential: - radosbench: clients: - client.0 concurrency: 128 size: 8192 time: 90 - radosbench: clients: - client.0 concurrency: 128 size: 8192 time: 90 - radosbench: clients: - client.0 concurrency: 128 size: 8192 time: 90 - radosbench: clients: - client.0 concurrency: 128 size: 8192 time: 90 - radosbench: clients: - client.0 concurrency: 128 size: 8192 time: 90 - radosbench: clients: - client.0 concurrency: 128 size: 8192 time: 90 - radosbench: clients: - client.0 concurrency: 128 size: 8192 time: 90 - radosbench: clients: - client.0 concurrency: 128 size: 8192 time: 90 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-23_01:23:55 tube: smithi user: yingxin verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-23T01:35:27.637 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4/qa; will attempt to use it 2024-04-23T01:35:27.638 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4/qa/tasks 2024-04-23T01:35:27.638 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-23T01:35:27.640 INFO:teuthology.task.internal:Checking packages... 2024-04-23T01:35:27.665 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'crimson' and ceph hash 'a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4' 2024-04-23T01:35:27.665 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-23T01:35:27.665 INFO:teuthology.packaging:ref: None 2024-04-23T01:35:27.665 INFO:teuthology.packaging:tag: None 2024-04-23T01:35:27.665 INFO:teuthology.packaging:branch: ci-yingxin-crimson-fix-tri-mutex-4 2024-04-23T01:35:27.665 INFO:teuthology.packaging:sha1: a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4 2024-04-23T01:35:27.665 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&ref=ci-yingxin-crimson-fix-tri-mutex-4 2024-04-23T01:35:27.812 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3192.ga1f34943 2024-04-23T01:35:27.814 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-23T01:35:27.821 INFO:teuthology.task.internal:no buildpackages task found 2024-04-23T01:35:27.821 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-23T01:35:27.858 INFO:teuthology.task.internal:Saving configuration 2024-04-23T01:35:27.876 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-23T01:35:27.882 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T01:35:27.909 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi090.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yingxin-2024-04-23_01:23:55-crimson-rados-ci-yingxin-crimson-fix-tri-mutex-4-distro-default-smithi/7669459', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 01:29:42.256351', 'locked_by': 'scheduled_yingxin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAWC5iJNafd1R+E2MWb4IxaYKCxUJ+HluqBuYCuGUXLeh2aLSeliTX1hcdEivTGrHmcBSK7uu5wXYfheVEoQH4I='} 2024-04-23T01:35:27.932 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi113.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yingxin-2024-04-23_01:23:55-crimson-rados-ci-yingxin-crimson-fix-tri-mutex-4-distro-default-smithi/7669459', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 01:29:42.254792', 'locked_by': 'scheduled_yingxin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCKdEn9EAfV/3De7+BKt/+rJPlg1u5r1QRzsikh2mPtC8ERZ/gQkM/mGeLMZrXk30QriId993Gxm6p+dUkuqAB4='} 2024-04-23T01:35:27.933 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-23T01:35:27.939 INFO:teuthology.task.internal:roles: ubuntu@smithi090.front.sepia.ceph.com - ['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'] 2024-04-23T01:35:27.939 INFO:teuthology.task.internal:roles: ubuntu@smithi113.front.sepia.ceph.com - ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-04-23T01:35:27.939 INFO:teuthology.run_tasks:Running task console_log... 2024-04-23T01:35:28.033 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f35cd98ce50>, signals=[15]) 2024-04-23T01:35:28.034 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-23T01:35:28.042 INFO:teuthology.task.internal:Opening connections... 2024-04-23T01:35:28.042 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi090.front.sepia.ceph.com 2024-04-23T01:35:28.045 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T01:35:28.121 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi113.front.sepia.ceph.com 2024-04-23T01:35:28.122 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T01:35:28.197 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-23T01:35:28.205 DEBUG:teuthology.orchestra.run.smithi090:> uname -m 2024-04-23T01:35:28.223 INFO:teuthology.orchestra.run.smithi090.stdout:x86_64 2024-04-23T01:35:28.223 DEBUG:teuthology.orchestra.run.smithi090:> cat /etc/os-release 2024-04-23T01:35:28.280 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="CentOS Stream" 2024-04-23T01:35:28.281 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="9" 2024-04-23T01:35:28.281 INFO:teuthology.orchestra.run.smithi090.stdout:ID="centos" 2024-04-23T01:35:28.281 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE="rhel fedora" 2024-04-23T01:35:28.281 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="9" 2024-04-23T01:35:28.281 INFO:teuthology.orchestra.run.smithi090.stdout:PLATFORM_ID="platform:el9" 2024-04-23T01:35:28.281 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-23T01:35:28.281 INFO:teuthology.orchestra.run.smithi090.stdout:ANSI_COLOR="0;31" 2024-04-23T01:35:28.281 INFO:teuthology.orchestra.run.smithi090.stdout:LOGO="fedora-logo-icon" 2024-04-23T01:35:28.281 INFO:teuthology.orchestra.run.smithi090.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-23T01:35:28.281 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://centos.org/" 2024-04-23T01:35:28.281 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-23T01:35:28.281 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-23T01:35:28.282 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-23T01:35:28.282 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server 2024-04-23T01:35:28.302 DEBUG:teuthology.orchestra.run.smithi113:> uname -m 2024-04-23T01:35:28.321 INFO:teuthology.orchestra.run.smithi113.stdout:x86_64 2024-04-23T01:35:28.321 DEBUG:teuthology.orchestra.run.smithi113:> cat /etc/os-release 2024-04-23T01:35:28.378 INFO:teuthology.orchestra.run.smithi113.stdout:NAME="CentOS Stream" 2024-04-23T01:35:28.378 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION="9" 2024-04-23T01:35:28.379 INFO:teuthology.orchestra.run.smithi113.stdout:ID="centos" 2024-04-23T01:35:28.379 INFO:teuthology.orchestra.run.smithi113.stdout:ID_LIKE="rhel fedora" 2024-04-23T01:35:28.379 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION_ID="9" 2024-04-23T01:35:28.379 INFO:teuthology.orchestra.run.smithi113.stdout:PLATFORM_ID="platform:el9" 2024-04-23T01:35:28.379 INFO:teuthology.orchestra.run.smithi113.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-23T01:35:28.379 INFO:teuthology.orchestra.run.smithi113.stdout:ANSI_COLOR="0;31" 2024-04-23T01:35:28.379 INFO:teuthology.orchestra.run.smithi113.stdout:LOGO="fedora-logo-icon" 2024-04-23T01:35:28.379 INFO:teuthology.orchestra.run.smithi113.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-23T01:35:28.379 INFO:teuthology.orchestra.run.smithi113.stdout:HOME_URL="https://centos.org/" 2024-04-23T01:35:28.379 INFO:teuthology.orchestra.run.smithi113.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-23T01:35:28.379 INFO:teuthology.orchestra.run.smithi113.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-23T01:35:28.380 INFO:teuthology.orchestra.run.smithi113.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-23T01:35:28.380 INFO:teuthology.lock.ops:Updating smithi113.front.sepia.ceph.com on lock server 2024-04-23T01:35:28.410 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-23T01:35:28.419 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-23T01:35:28.449 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-23T01:35:28.450 DEBUG:teuthology.orchestra.run.smithi090:> test '!' -e /home/ubuntu/cephtest 2024-04-23T01:35:28.452 DEBUG:teuthology.orchestra.run.smithi113:> test '!' -e /home/ubuntu/cephtest 2024-04-23T01:35:28.469 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-23T01:35:28.474 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-23T01:35:28.474 DEBUG:teuthology.orchestra.run.smithi090:> test -z $(ls -A /var/lib/ceph) 2024-04-23T01:35:28.510 DEBUG:teuthology.orchestra.run.smithi113:> test -z $(ls -A /var/lib/ceph) 2024-04-23T01:35:28.618 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-23T01:35:28.665 INFO:teuthology.run_tasks:Running task kernel... 2024-04-23T01:35:28.680 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-23T01:35:28.681 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-23T01:35:28.681 DEBUG:teuthology.orchestra.run.smithi090:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T01:35:28.681 DEBUG:teuthology.orchestra.run.smithi113:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T01:35:28.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T01:35:28.698 DEBUG:teuthology.orchestra.run.smithi090:> uname -r 2024-04-23T01:35:28.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T01:35:28.699 DEBUG:teuthology.orchestra.run.smithi113:> uname -r 2024-04-23T01:35:28.755 INFO:teuthology.orchestra.run.smithi090.stdout:5.14.0-437.el9.x86_64 2024-04-23T01:35:28.755 INFO:teuthology.task.kernel:Running kernel on smithi090: 5.14.0-437.el9.x86_64 2024-04-23T01:35:28.756 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum install -y kernel 2024-04-23T01:35:28.757 INFO:teuthology.orchestra.run.smithi113.stdout:5.14.0-437.el9.x86_64 2024-04-23T01:35:28.758 INFO:teuthology.task.kernel:Running kernel on smithi113: 5.14.0-437.el9.x86_64 2024-04-23T01:35:28.758 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum install -y kernel 2024-04-23T01:36:37.904 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - BaseOS 122 kB/s | 8.0 MB 01:07 2024-04-23T01:36:40.380 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - BaseOS 118 kB/s | 8.0 MB 01:09 2024-04-23T01:36:42.947 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - AppStream 4.8 MB/s | 19 MB 00:03 2024-04-23T01:36:50.489 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - CRB 1.6 MB/s | 6.2 MB 00:03 2024-04-23T01:36:51.929 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 16 kB 00:00 2024-04-23T01:36:53.424 INFO:teuthology.orchestra.run.smithi113.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-23T01:36:57.044 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:15 2024-04-23T01:36:58.354 INFO:teuthology.orchestra.run.smithi113.stdout:lab-extras 46 kB/s | 1.7 kB 00:00 2024-04-23T01:37:00.194 INFO:teuthology.orchestra.run.smithi113.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-23T01:37:00.194 INFO:teuthology.orchestra.run.smithi113.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-23T01:37:00.290 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-23T01:37:00.291 INFO:teuthology.orchestra.run.smithi113.stdout:Nothing to do. 2024-04-23T01:37:00.291 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-23T01:37:00.428 DEBUG:teuthology.orchestra.run.smithi113:> echo no | sudo yum reinstall kernel || true 2024-04-23T01:37:00.922 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:00:02 ago on Tue 23 Apr 2024 01:36:58 AM UTC. 2024-04-23T01:37:01.051 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-23T01:37:01.063 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:37:01.063 INFO:teuthology.orchestra.run.smithi113.stdout: Package Architecture Version Repository Size 2024-04-23T01:37:01.063 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:37:01.064 INFO:teuthology.orchestra.run.smithi113.stdout:Reinstalling: 2024-04-23T01:37:01.064 INFO:teuthology.orchestra.run.smithi113.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-23T01:37:01.064 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:37:01.064 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-04-23T01:37:01.064 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:37:01.064 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:37:01.064 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 255 k 2024-04-23T01:37:01.064 INFO:teuthology.orchestra.run.smithi113.stdout:Installed size: 0 2024-04-23T01:37:01.064 INFO:teuthology.orchestra.run.smithi113.stderr:Operation aborted. 2024-04-23T01:37:01.125 INFO:teuthology.orchestra.run.smithi113.stdout:Is this ok [y/N]: 2024-04-23T01:37:01.126 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum reinstall -y kernel || true 2024-04-23T01:37:01.619 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:00:03 ago on Tue 23 Apr 2024 01:36:58 AM UTC. 2024-04-23T01:37:01.735 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-23T01:37:01.736 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:37:01.736 INFO:teuthology.orchestra.run.smithi113.stdout: Package Architecture Version Repository Size 2024-04-23T01:37:01.736 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:37:01.736 INFO:teuthology.orchestra.run.smithi113.stdout:Reinstalling: 2024-04-23T01:37:01.736 INFO:teuthology.orchestra.run.smithi113.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-23T01:37:01.736 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:37:01.736 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-04-23T01:37:01.736 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:37:01.736 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:37:01.736 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 255 k 2024-04-23T01:37:01.736 INFO:teuthology.orchestra.run.smithi113.stdout:Installed size: 0 2024-04-23T01:37:01.736 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2024-04-23T01:37:02.265 INFO:teuthology.orchestra.run.smithi113.stdout:kernel-5.14.0-437.el9.x86_64.rpm 951 kB/s | 255 kB 00:00 2024-04-23T01:37:02.265 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:37:02.265 INFO:teuthology.orchestra.run.smithi113.stdout:Total 482 kB/s | 255 kB 00:00 2024-04-23T01:37:02.282 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2024-04-23T01:37:02.461 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2024-04-23T01:37:02.462 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2024-04-23T01:37:02.466 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2024-04-23T01:37:02.467 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2024-04-23T01:37:02.614 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2024-04-23T01:37:02.691 INFO:teuthology.orchestra.run.smithi113.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-23T01:37:02.691 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T01:37:03.360 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T01:37:03.360 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-23T01:37:04.077 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T01:37:04.078 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:37:04.078 INFO:teuthology.orchestra.run.smithi113.stdout:Reinstalled: 2024-04-23T01:37:04.078 INFO:teuthology.orchestra.run.smithi113.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-23T01:37:04.078 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:37:04.078 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-23T01:37:04.244 DEBUG:teuthology.orchestra.run.smithi113:> rpm -q kernel | sort -rV | head -n 1 2024-04-23T01:37:04.290 INFO:teuthology.orchestra.run.smithi113.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-23T01:37:04.291 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-23T01:37:04.291 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-23T01:37:04.291 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-04-23T01:37:04.291 DEBUG:teuthology.orchestra.run.smithi113:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-23T01:37:04.332 INFO:teuthology.orchestra.run.smithi113.stdout:ttyS1 2024-04-23T01:37:04.356 DEBUG:teuthology.parallel:result is None 2024-04-23T01:37:06.075 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - CRB 1.1 MB/s | 6.2 MB 00:05 2024-04-23T01:37:07.648 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 16 kB 00:00 2024-04-23T01:37:13.623 INFO:teuthology.orchestra.run.smithi090.stdout:Extra Packages for Enterprise Linux 3.6 MB/s | 21 MB 00:05 2024-04-23T01:37:18.602 INFO:teuthology.orchestra.run.smithi090.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-23T01:37:20.508 INFO:teuthology.orchestra.run.smithi090.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-23T01:37:20.508 INFO:teuthology.orchestra.run.smithi090.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-23T01:37:20.616 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-23T01:37:20.618 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-23T01:37:20.618 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-23T01:37:20.720 DEBUG:teuthology.orchestra.run.smithi090:> echo no | sudo yum reinstall kernel || true 2024-04-23T01:37:21.219 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:00:03 ago on Tue 23 Apr 2024 01:37:18 AM UTC. 2024-04-23T01:37:21.342 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-23T01:37:21.360 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:37:21.360 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2024-04-23T01:37:21.360 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:37:21.360 INFO:teuthology.orchestra.run.smithi090.stdout:Reinstalling: 2024-04-23T01:37:21.360 INFO:teuthology.orchestra.run.smithi090.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-23T01:37:21.360 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:37:21.360 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-23T01:37:21.360 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:37:21.360 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:37:21.360 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 255 k 2024-04-23T01:37:21.361 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 0 2024-04-23T01:37:21.361 INFO:teuthology.orchestra.run.smithi090.stderr:Operation aborted. 2024-04-23T01:37:21.448 INFO:teuthology.orchestra.run.smithi090.stdout:Is this ok [y/N]: 2024-04-23T01:37:21.448 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum reinstall -y kernel || true 2024-04-23T01:37:21.940 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:00:03 ago on Tue 23 Apr 2024 01:37:18 AM UTC. 2024-04-23T01:37:22.055 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-23T01:37:22.056 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:37:22.056 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2024-04-23T01:37:22.056 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:37:22.056 INFO:teuthology.orchestra.run.smithi090.stdout:Reinstalling: 2024-04-23T01:37:22.056 INFO:teuthology.orchestra.run.smithi090.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-23T01:37:22.056 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:37:22.057 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-23T01:37:22.057 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:37:22.057 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:37:22.057 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 255 k 2024-04-23T01:37:22.057 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 0 2024-04-23T01:37:22.057 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-23T01:37:23.171 INFO:teuthology.orchestra.run.smithi090.stdout:kernel-5.14.0-437.el9.x86_64.rpm 266 kB/s | 255 kB 00:00 2024-04-23T01:37:23.171 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:37:23.172 INFO:teuthology.orchestra.run.smithi090.stdout:Total 229 kB/s | 255 kB 00:01 2024-04-23T01:37:23.192 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-23T01:37:23.403 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-23T01:37:23.403 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-23T01:37:23.408 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-23T01:37:23.409 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-23T01:37:23.548 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-23T01:37:23.631 INFO:teuthology.orchestra.run.smithi090.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-23T01:37:23.631 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T01:37:24.424 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T01:37:24.425 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-23T01:37:25.208 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T01:37:25.208 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:37:25.209 INFO:teuthology.orchestra.run.smithi090.stdout:Reinstalled: 2024-04-23T01:37:25.209 INFO:teuthology.orchestra.run.smithi090.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-23T01:37:25.209 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:37:25.209 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-23T01:37:25.414 DEBUG:teuthology.orchestra.run.smithi090:> rpm -q kernel | sort -rV | head -n 1 2024-04-23T01:37:25.460 INFO:teuthology.orchestra.run.smithi090.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-23T01:37:25.461 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-23T01:37:25.461 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-23T01:37:25.461 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-23T01:37:25.461 DEBUG:teuthology.orchestra.run.smithi090:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-23T01:37:25.494 INFO:teuthology.orchestra.run.smithi090.stdout:ttyS1 2024-04-23T01:37:25.515 DEBUG:teuthology.parallel:result is None 2024-04-23T01:37:25.516 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-23T01:37:25.523 INFO:teuthology.task.internal:Creating test directory... 2024-04-23T01:37:25.523 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T01:37:25.558 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T01:37:25.576 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-23T01:37:25.584 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-23T01:37:25.603 INFO:teuthology.task.internal:Creating archive directory... 2024-04-23T01:37:25.604 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T01:37:25.613 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T01:37:25.642 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-23T01:37:25.649 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-23T01:37:25.650 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T01:37:25.675 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T01:37:25.698 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T01:37:25.707 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T01:37:25.713 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T01:37:25.724 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T01:37:25.726 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-23T01:37:25.733 INFO:teuthology.task.internal:Configuring sudo... 2024-04-23T01:37:25.733 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T01:37:25.751 DEBUG:teuthology.orchestra.run.smithi113:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T01:37:25.793 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-23T01:37:25.802 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-23T01:37:25.802 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T01:37:25.814 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T01:37:25.849 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T01:37:25.909 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T01:37:25.953 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T01:37:26.034 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T01:37:26.072 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-23T01:37:26.072 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T01:37:26.135 DEBUG:teuthology.orchestra.run.smithi113:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T01:37:26.180 DEBUG:teuthology.orchestra.run.smithi113:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T01:37:26.228 DEBUG:teuthology.orchestra.run.smithi113:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T01:37:26.312 DEBUG:teuthology.orchestra.run.smithi113:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T01:37:26.352 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-23T01:37:26.353 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T01:37:26.417 DEBUG:teuthology.orchestra.run.smithi090:> sudo service rsyslog restart 2024-04-23T01:37:26.420 DEBUG:teuthology.orchestra.run.smithi113:> sudo service rsyslog restart 2024-04-23T01:37:26.451 INFO:teuthology.orchestra.run.smithi090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T01:37:26.492 INFO:teuthology.orchestra.run.smithi113.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T01:37:27.000 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-23T01:37:27.008 INFO:teuthology.task.internal:Starting timer... 2024-04-23T01:37:27.008 INFO:teuthology.run_tasks:Running task pcp... 2024-04-23T01:37:27.032 INFO:teuthology.run_tasks:Running task selinux... 2024-04-23T01:37:27.064 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-23T01:37:27.064 DEBUG:teuthology.orchestra.run.smithi090:> sudo service auditd rotate 2024-04-23T01:37:27.122 INFO:teuthology.orchestra.run.smithi090.stdout:Rotating logs: 2024-04-23T01:37:27.124 DEBUG:teuthology.orchestra.run.smithi113:> sudo service auditd rotate 2024-04-23T01:37:27.188 INFO:teuthology.orchestra.run.smithi113.stdout:Rotating logs: 2024-04-23T01:37:27.190 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-23T01:37:27.190 DEBUG:teuthology.orchestra.run.smithi090:> /usr/sbin/getenforce 2024-04-23T01:37:27.212 INFO:teuthology.orchestra.run.smithi090.stdout:Permissive 2024-04-23T01:37:27.212 DEBUG:teuthology.orchestra.run.smithi113:> /usr/sbin/getenforce 2024-04-23T01:37:27.237 INFO:teuthology.orchestra.run.smithi113.stdout:Permissive 2024-04-23T01:37:27.238 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi090.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi113.front.sepia.ceph.com': 'permissive'} 2024-04-23T01:37:27.238 DEBUG:teuthology.orchestra.run.smithi090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T01:37:27.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T01:37:27.276 DEBUG:teuthology.orchestra.run.smithi113:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T01:37:27.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T01:37:27.302 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-23T01:37:27.302 DEBUG:teuthology.orchestra.run.smithi090:> sudo /usr/sbin/setenforce permissive 2024-04-23T01:37:27.347 DEBUG:teuthology.orchestra.run.smithi113:> sudo /usr/sbin/setenforce permissive 2024-04-23T01:37:27.373 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-23T01:37:27.383 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-23T01:37:27.399 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-23T01:37:27.401 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi090.front.sepia.ceph.com,smithi113.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-23T01:40:55.447 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi090.front.sepia.ceph.com'), Remote(name='ubuntu@smithi113.front.sepia.ceph.com')] 2024-04-23T01:40:55.448 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-04-23T01:40:55.449 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T01:40:55.522 DEBUG:teuthology.orchestra.run.smithi090:> true 2024-04-23T01:40:55.601 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-04-23T01:40:55.601 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-04-23T01:40:55.601 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T01:40:55.676 DEBUG:teuthology.orchestra.run.smithi113:> true 2024-04-23T01:40:55.761 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-04-23T01:40:55.762 INFO:teuthology.run_tasks:Running task clock... 2024-04-23T01:40:55.772 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-23T01:40:55.773 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T01:40:55.773 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T01:40:55.775 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T01:40:55.776 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T01:40:55.799 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-23T01:40:55.812 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-23T01:40:55.829 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-23T01:40:55.834 INFO:teuthology.orchestra.run.smithi090.stderr:sudo: ntpd: command not found 2024-04-23T01:40:55.842 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-23T01:40:55.844 INFO:teuthology.orchestra.run.smithi090.stdout:506 Cannot talk to daemon 2024-04-23T01:40:55.857 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-23T01:40:55.865 INFO:teuthology.orchestra.run.smithi113.stderr:sudo: ntpd: command not found 2024-04-23T01:40:55.870 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-23T01:40:55.876 INFO:teuthology.orchestra.run.smithi113.stdout:506 Cannot talk to daemon 2024-04-23T01:40:55.889 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-23T01:40:55.901 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-23T01:40:55.924 INFO:teuthology.orchestra.run.smithi090.stderr:bash: line 1: ntpq: command not found 2024-04-23T01:40:55.927 INFO:teuthology.orchestra.run.smithi090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T01:40:55.927 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================== 2024-04-23T01:40:55.927 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T01:40:55.927 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T01:40:55.927 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T01:40:55.927 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T01:40:55.957 INFO:teuthology.orchestra.run.smithi113.stderr:bash: line 1: ntpq: command not found 2024-04-23T01:40:55.960 INFO:teuthology.orchestra.run.smithi113.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T01:40:55.960 INFO:teuthology.orchestra.run.smithi113.stdout:=============================================================================== 2024-04-23T01:40:55.960 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T01:40:55.960 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T01:40:55.960 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T01:40:55.960 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T01:40:55.961 INFO:teuthology.run_tasks:Running task install... 2024-04-23T01:40:55.969 DEBUG:teuthology.task.install:project ceph 2024-04-23T01:40:55.969 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'crimson', 'sha1': 'a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4'}} 2024-04-23T01:40:55.969 DEBUG:teuthology.task.install:config {'flavor': 'crimson', 'sha1': 'a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4'} 2024-04-23T01:40:55.969 INFO:teuthology.task.install:Using flavor: crimson 2024-04-23T01:40:55.975 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-23T01:40:56.029 INFO:teuthology.task.install:extra packages: [] 2024-04-23T01:40:56.029 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4', 'tag': None, 'wait_for_package': False} 2024-04-23T01:40:56.030 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4 2024-04-23T01:40:56.032 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4', 'tag': None, 'wait_for_package': False} 2024-04-23T01:40:56.032 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4 2024-04-23T01:40:56.176 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/ci-yingxin-crimson-fix-tri-mutex-4/a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4/centos/9/flavors/crimson/ 2024-04-23T01:40:56.177 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3192.ga1f34943 2024-04-23T01:40:56.180 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/ci-yingxin-crimson-fix-tri-mutex-4/a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4/centos/9/flavors/crimson/ 2024-04-23T01:40:56.181 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3192.ga1f34943 2024-04-23T01:40:56.296 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/ci-yingxin-crimson-fix-tri-mutex-4/a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4/centos/9/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/ci-yingxin-crimson-fix-tri-mutex-4/a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4/centos/9/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/ci-yingxin-crimson-fix-tri-mutex-4/a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4/centos/9/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-23T01:40:56.296 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-23T01:40:56.296 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-23T01:40:56.321 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-23T01:40:56.321 DEBUG:teuthology.orchestra.run.smithi113:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-23T01:40:56.352 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/ci-yingxin-crimson-fix-tri-mutex-4/a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4/centos/9/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/ci-yingxin-crimson-fix-tri-mutex-4/a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4/centos/9/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/ci-yingxin-crimson-fix-tri-mutex-4/a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4/centos/9/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-23T01:40:56.353 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-23T01:40:56.353 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-23T01:40:56.378 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-23T01:40:56.378 DEBUG:teuthology.orchestra.run.smithi090:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-23T01:40:56.385 DEBUG:teuthology.orchestra.run.smithi113:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-23T01:40:56.444 DEBUG:teuthology.orchestra.run.smithi090:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-23T01:40:56.458 DEBUG:teuthology.orchestra.run.smithi113:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-23T01:40:56.517 DEBUG:teuthology.orchestra.run.smithi090:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-23T01:40:56.521 INFO:teuthology.orchestra.run.smithi113.stdout:check_obsoletes = 1 2024-04-23T01:40:56.522 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum clean all 2024-04-23T01:40:56.580 INFO:teuthology.orchestra.run.smithi090.stdout:check_obsoletes = 1 2024-04-23T01:40:56.581 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum clean all 2024-04-23T01:40:56.745 INFO:teuthology.orchestra.run.smithi113.stdout:48 files removed 2024-04-23T01:40:56.772 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install ceph-radosgw 2024-04-23T01:40:56.806 INFO:teuthology.orchestra.run.smithi090.stdout:48 files removed 2024-04-23T01:40:56.833 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-radosgw 2024-04-23T01:40:57.698 INFO:teuthology.orchestra.run.smithi090.stdout:ceph packages for x86_64 144 kB/s | 87 kB 00:00 2024-04-23T01:40:57.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph packages for x86_64 104 kB/s | 87 kB 00:00 2024-04-23T01:40:58.124 INFO:teuthology.orchestra.run.smithi090.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-04-23T01:40:58.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-04-23T01:40:58.475 INFO:teuthology.orchestra.run.smithi090.stdout:ceph source packages 6.5 kB/s | 2.0 kB 00:00 2024-04-23T01:40:58.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph source packages 5.4 kB/s | 2.0 kB 00:00 2024-04-23T01:41:02.196 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - BaseOS 2.2 MB/s | 8.0 MB 00:03 2024-04-23T01:41:05.512 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - BaseOS 1.2 MB/s | 8.0 MB 00:06 2024-04-23T01:41:09.500 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - AppStream 3.1 MB/s | 19 MB 00:06 2024-04-23T01:41:15.501 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - AppStream 2.1 MB/s | 19 MB 00:08 2024-04-23T01:41:18.195 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - CRB 1.2 MB/s | 6.2 MB 00:05 2024-04-23T01:41:19.770 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-04-23T01:41:22.471 INFO:teuthology.orchestra.run.smithi090.stdout:Extra Packages for Enterprise Linux 8.1 MB/s | 21 MB 00:02 2024-04-23T01:41:23.289 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - CRB 1.5 MB/s | 6.2 MB 00:04 2024-04-23T01:41:25.119 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - Extras packages 23 kB/s | 16 kB 00:00 2024-04-23T01:41:26.779 INFO:teuthology.orchestra.run.smithi113.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-23T01:41:27.514 INFO:teuthology.orchestra.run.smithi090.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-04-23T01:41:29.622 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-23T01:41:29.624 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:41:29.624 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-04-23T01:41:29.624 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:41:29.624 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-23T01:41:29.625 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-radosgw x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 10 M 2024-04-23T01:41:29.625 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrading: 2024-04-23T01:41:29.625 INFO:teuthology.orchestra.run.smithi090.stdout: librados2 x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 3.4 M 2024-04-23T01:41:29.625 INFO:teuthology.orchestra.run.smithi090.stdout: librbd1 x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 3.7 M 2024-04-23T01:41:29.625 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-04-23T01:41:29.625 INFO:teuthology.orchestra.run.smithi090.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-23T01:41:29.625 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 5.9 M 2024-04-23T01:41:29.625 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-common x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 19 M 2024-04-23T01:41:29.625 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-selinux x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 25 k 2024-04-23T01:41:29.625 INFO:teuthology.orchestra.run.smithi090.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-23T01:41:29.625 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-23T01:41:29.625 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-23T01:41:29.625 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2 x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 796 k 2024-04-23T01:41:29.625 INFO:teuthology.orchestra.run.smithi090.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-23T01:41:29.626 INFO:teuthology.orchestra.run.smithi090.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-23T01:41:29.626 INFO:teuthology.orchestra.run.smithi090.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-23T01:41:29.626 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1 x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 548 k 2024-04-23T01:41:29.626 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-23T01:41:29.626 INFO:teuthology.orchestra.run.smithi090.stdout: librgw2 x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 5.3 M 2024-04-23T01:41:29.626 INFO:teuthology.orchestra.run.smithi090.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-23T01:41:29.626 INFO:teuthology.orchestra.run.smithi090.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-23T01:41:29.626 INFO:teuthology.orchestra.run.smithi090.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-23T01:41:29.626 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 45 k 2024-04-23T01:41:29.626 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-common x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 135 k 2024-04-23T01:41:29.626 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cephfs x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 159 k 2024-04-23T01:41:29.626 INFO:teuthology.orchestra.run.smithi090.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-23T01:41:29.627 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 315 k 2024-04-23T01:41:29.627 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rbd x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 293 k 2024-04-23T01:41:29.627 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rgw x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 99 k 2024-04-23T01:41:29.627 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-23T01:41:29.627 INFO:teuthology.orchestra.run.smithi090.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-23T01:41:29.627 INFO:teuthology.orchestra.run.smithi090.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-23T01:41:29.627 INFO:teuthology.orchestra.run.smithi090.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-23T01:41:29.627 INFO:teuthology.orchestra.run.smithi090.stdout:Installing weak dependencies: 2024-04-23T01:41:29.627 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-23T01:41:29.627 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:41:29.627 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-23T01:41:29.627 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:41:29.627 INFO:teuthology.orchestra.run.smithi090.stdout:Install 30 Packages 2024-04-23T01:41:29.628 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrade 2 Packages 2024-04-23T01:41:29.628 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:41:29.628 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 59 M 2024-04-23T01:41:29.628 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-23T01:41:30.791 INFO:teuthology.orchestra.run.smithi090.stdout:(1/32): ceph-radosgw-19.0.0-3192.ga1f34943.el7. 12 MB/s | 10 MB 00:00 2024-04-23T01:41:30.833 INFO:teuthology.orchestra.run.smithi090.stdout:(2/32): ceph-selinux-19.0.0-3192.ga1f34943.el7. 606 kB/s | 25 kB 00:00 2024-04-23T01:41:30.908 INFO:teuthology.orchestra.run.smithi090.stdout:(3/32): libcephfs2-19.0.0-3192.ga1f34943.el7.x8 10 MB/s | 796 kB 00:00 2024-04-23T01:41:30.975 INFO:teuthology.orchestra.run.smithi090.stdout:(4/32): libradosstriper1-19.0.0-3192.ga1f34943. 8.0 MB/s | 548 kB 00:00 2024-04-23T01:41:31.283 INFO:teuthology.orchestra.run.smithi090.stdout:(5/32): librgw2-19.0.0-3192.ga1f34943.el7.x86_6 17 MB/s | 5.3 MB 00:00 2024-04-23T01:41:31.325 INFO:teuthology.orchestra.run.smithi090.stdout:(6/32): python3-ceph-argparse-19.0.0-3192.ga1f3 1.1 MB/s | 45 kB 00:00 2024-04-23T01:41:31.376 INFO:teuthology.orchestra.run.smithi090.stdout:(7/32): python3-ceph-common-19.0.0-3192.ga1f349 2.6 MB/s | 135 kB 00:00 2024-04-23T01:41:31.426 INFO:teuthology.orchestra.run.smithi090.stdout:(8/32): python3-cephfs-19.0.0-3192.ga1f34943.el 3.1 MB/s | 159 kB 00:00 2024-04-23T01:41:31.485 INFO:teuthology.orchestra.run.smithi090.stdout:(9/32): python3-rados-19.0.0-3192.ga1f34943.el7 5.3 MB/s | 315 kB 00:00 2024-04-23T01:41:31.536 INFO:teuthology.orchestra.run.smithi090.stdout:(10/32): python3-rbd-19.0.0-3192.ga1f34943.el7. 5.6 MB/s | 293 kB 00:00 2024-04-23T01:41:31.587 INFO:teuthology.orchestra.run.smithi090.stdout:(11/32): python3-rgw-19.0.0-3192.ga1f34943.el7. 1.9 MB/s | 99 kB 00:00 2024-04-23T01:41:31.912 INFO:teuthology.orchestra.run.smithi113.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-04-23T01:41:31.912 INFO:teuthology.orchestra.run.smithi090.stdout:(12/32): boost-program-options-1.75.0-8.el9.x86 329 kB/s | 107 kB 00:00 2024-04-23T01:41:32.045 INFO:teuthology.orchestra.run.smithi090.stdout:(13/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-04-23T01:41:32.121 INFO:teuthology.orchestra.run.smithi090.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 603 kB/s | 45 kB 00:00 2024-04-23T01:41:32.296 INFO:teuthology.orchestra.run.smithi090.stdout:(15/32): librdkafka-1.6.1-102.el9.x86_64.rpm 3.7 MB/s | 662 kB 00:00 2024-04-23T01:41:32.372 INFO:teuthology.orchestra.run.smithi090.stdout:(16/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-04-23T01:41:32.472 INFO:teuthology.orchestra.run.smithi090.stdout:(17/32): ceph-base-19.0.0-3192.ga1f34943.el7.x8 2.3 MB/s | 5.9 MB 00:02 2024-04-23T01:41:32.497 INFO:teuthology.orchestra.run.smithi090.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 337 kB/s | 42 kB 00:00 2024-04-23T01:41:32.572 INFO:teuthology.orchestra.run.smithi090.stdout:(19/32): qatlib-service-24.02.0-1.el9.x86_64.rp 477 kB/s | 36 kB 00:00 2024-04-23T01:41:32.715 INFO:teuthology.orchestra.run.smithi090.stdout:(20/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 334 kB/s | 47 kB 00:00 2024-04-23T01:41:32.807 INFO:teuthology.orchestra.run.smithi090.stdout:(21/32): qatlib-24.02.0-1.el9.x86_64.rpm 660 kB/s | 221 kB 00:00 2024-04-23T01:41:33.366 INFO:teuthology.orchestra.run.smithi090.stdout:(22/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 475 kB/s | 309 kB 00:00 2024-04-23T01:41:33.474 INFO:teuthology.orchestra.run.smithi090.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 240 kB/s | 26 kB 00:00 2024-04-23T01:41:33.583 INFO:teuthology.orchestra.run.smithi090.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 449 kB/s | 49 kB 00:00 2024-04-23T01:41:33.691 INFO:teuthology.orchestra.run.smithi090.stdout:(25/32): libunwind-1.6.2-1.el9.x86_64.rpm 623 kB/s | 67 kB 00:00 2024-04-23T01:41:33.809 INFO:teuthology.orchestra.run.smithi090.stdout:(26/32): libarrow-9.0.0-10.el9.x86_64.rpm 4.4 MB/s | 4.4 MB 00:01 2024-04-23T01:41:33.909 INFO:teuthology.orchestra.run.smithi090.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-23T01:41:33.968 INFO:teuthology.orchestra.run.smithi090.stdout:(28/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.0 MB/s | 838 kB 00:00 2024-04-23T01:41:34.043 INFO:teuthology.orchestra.run.smithi090.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-23T01:41:34.043 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-23T01:41:34.045 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:41:34.045 INFO:teuthology.orchestra.run.smithi113.stdout: Package Arch Version Repository Size 2024-04-23T01:41:34.046 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:41:34.046 INFO:teuthology.orchestra.run.smithi113.stdout:Installing: 2024-04-23T01:41:34.046 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-radosgw x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 10 M 2024-04-23T01:41:34.046 INFO:teuthology.orchestra.run.smithi113.stdout:Upgrading: 2024-04-23T01:41:34.046 INFO:teuthology.orchestra.run.smithi113.stdout: librados2 x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 3.4 M 2024-04-23T01:41:34.046 INFO:teuthology.orchestra.run.smithi113.stdout: librbd1 x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 3.7 M 2024-04-23T01:41:34.046 INFO:teuthology.orchestra.run.smithi113.stdout:Installing dependencies: 2024-04-23T01:41:34.047 INFO:teuthology.orchestra.run.smithi113.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-23T01:41:34.047 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-base x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 5.9 M 2024-04-23T01:41:34.047 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-common x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 19 M 2024-04-23T01:41:34.047 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-selinux x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 25 k 2024-04-23T01:41:34.047 INFO:teuthology.orchestra.run.smithi113.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-23T01:41:34.047 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-23T01:41:34.047 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-23T01:41:34.047 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs2 x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 796 k 2024-04-23T01:41:34.047 INFO:teuthology.orchestra.run.smithi113.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-23T01:41:34.048 INFO:teuthology.orchestra.run.smithi113.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-23T01:41:34.048 INFO:teuthology.orchestra.run.smithi113.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-23T01:41:34.048 INFO:teuthology.orchestra.run.smithi113.stdout: libradosstriper1 x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 548 k 2024-04-23T01:41:34.048 INFO:teuthology.orchestra.run.smithi113.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-23T01:41:34.048 INFO:teuthology.orchestra.run.smithi113.stdout: librgw2 x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 5.3 M 2024-04-23T01:41:34.048 INFO:teuthology.orchestra.run.smithi113.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-23T01:41:34.048 INFO:teuthology.orchestra.run.smithi113.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-23T01:41:34.048 INFO:teuthology.orchestra.run.smithi113.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-23T01:41:34.049 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-argparse x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 45 k 2024-04-23T01:41:34.049 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-common x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 135 k 2024-04-23T01:41:34.049 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cephfs x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 159 k 2024-04-23T01:41:34.049 INFO:teuthology.orchestra.run.smithi113.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-23T01:41:34.049 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rados x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 315 k 2024-04-23T01:41:34.049 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rbd x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 293 k 2024-04-23T01:41:34.049 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rgw x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 99 k 2024-04-23T01:41:34.049 INFO:teuthology.orchestra.run.smithi113.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-23T01:41:34.050 INFO:teuthology.orchestra.run.smithi113.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-23T01:41:34.050 INFO:teuthology.orchestra.run.smithi113.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-23T01:41:34.050 INFO:teuthology.orchestra.run.smithi113.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-23T01:41:34.050 INFO:teuthology.orchestra.run.smithi113.stdout:Installing weak dependencies: 2024-04-23T01:41:34.050 INFO:teuthology.orchestra.run.smithi113.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-23T01:41:34.050 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:41:34.050 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-04-23T01:41:34.050 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:41:34.051 INFO:teuthology.orchestra.run.smithi113.stdout:Install 30 Packages 2024-04-23T01:41:34.051 INFO:teuthology.orchestra.run.smithi113.stdout:Upgrade 2 Packages 2024-04-23T01:41:34.051 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:41:34.051 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 59 M 2024-04-23T01:41:34.051 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2024-04-23T01:41:34.478 INFO:teuthology.orchestra.run.smithi090.stdout:(30/32): librbd1-19.0.0-3192.ga1f34943.el7.x86_ 8.5 MB/s | 3.7 MB 00:00 2024-04-23T01:41:35.252 INFO:teuthology.orchestra.run.smithi090.stdout:(31/32): librados2-19.0.0-3192.ga1f34943.el7.x8 2.6 MB/s | 3.4 MB 00:01 2024-04-23T01:41:35.422 INFO:teuthology.orchestra.run.smithi113.stdout:(1/32): ceph-base-19.0.0-3192.ga1f34943.el7.x86 5.6 MB/s | 5.9 MB 00:01 2024-04-23T01:41:35.477 INFO:teuthology.orchestra.run.smithi090.stdout:(32/32): ceph-common-19.0.0-3192.ga1f34943.el7. 3.5 MB/s | 19 MB 00:05 2024-04-23T01:41:35.480 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:41:35.481 INFO:teuthology.orchestra.run.smithi090.stdout:Total 10 MB/s | 59 MB 00:05 2024-04-23T01:41:35.513 INFO:teuthology.orchestra.run.smithi113.stdout:(2/32): ceph-selinux-19.0.0-3192.ga1f34943.el7. 275 kB/s | 25 kB 00:00 2024-04-23T01:41:35.576 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-23T01:41:35.612 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-23T01:41:35.612 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-23T01:41:35.638 INFO:teuthology.orchestra.run.smithi113.stdout:(3/32): libcephfs2-19.0.0-3192.ga1f34943.el7.x8 6.2 MB/s | 796 kB 00:00 2024-04-23T01:41:35.739 INFO:teuthology.orchestra.run.smithi113.stdout:(4/32): libradosstriper1-19.0.0-3192.ga1f34943. 5.3 MB/s | 548 kB 00:00 2024-04-23T01:41:36.072 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-23T01:41:36.102 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-23T01:41:36.247 INFO:teuthology.orchestra.run.smithi113.stdout:(5/32): librgw2-19.0.0-3192.ga1f34943.el7.x86_6 10 MB/s | 5.3 MB 00:00 2024-04-23T01:41:36.364 INFO:teuthology.orchestra.run.smithi113.stdout:(6/32): ceph-radosgw-19.0.0-3192.ga1f34943.el7. 5.1 MB/s | 10 MB 00:01 2024-04-23T01:41:36.390 INFO:teuthology.orchestra.run.smithi113.stdout:(7/32): python3-ceph-argparse-19.0.0-3192.ga1f3 317 kB/s | 45 kB 00:00 2024-04-23T01:41:36.431 INFO:teuthology.orchestra.run.smithi113.stdout:(8/32): python3-ceph-common-19.0.0-3192.ga1f349 2.0 MB/s | 135 kB 00:00 2024-04-23T01:41:36.457 INFO:teuthology.orchestra.run.smithi113.stdout:(9/32): python3-cephfs-19.0.0-3192.ga1f34943.el 2.3 MB/s | 159 kB 00:00 2024-04-23T01:41:36.507 INFO:teuthology.orchestra.run.smithi113.stdout:(10/32): python3-rados-19.0.0-3192.ga1f34943.el 4.1 MB/s | 315 kB 00:00 2024-04-23T01:41:36.549 INFO:teuthology.orchestra.run.smithi113.stdout:(11/32): python3-rbd-19.0.0-3192.ga1f34943.el7. 3.1 MB/s | 293 kB 00:00 2024-04-23T01:41:36.582 INFO:teuthology.orchestra.run.smithi113.stdout:(12/32): python3-rgw-19.0.0-3192.ga1f34943.el7. 1.3 MB/s | 99 kB 00:00 2024-04-23T01:41:36.875 INFO:teuthology.orchestra.run.smithi113.stdout:(13/32): ceph-common-19.0.0-3192.ga1f34943.el7. 7.8 MB/s | 19 MB 00:02 2024-04-23T01:41:36.900 INFO:teuthology.orchestra.run.smithi113.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 304 kB/s | 107 kB 00:00 2024-04-23T01:41:36.959 INFO:teuthology.orchestra.run.smithi113.stdout:(15/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 426 kB/s | 160 kB 00:00 2024-04-23T01:41:37.026 INFO:teuthology.orchestra.run.smithi113.stdout:(16/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 301 kB/s | 45 kB 00:00 2024-04-23T01:41:37.051 INFO:teuthology.orchestra.run.smithi113.stdout:(17/32): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-04-23T01:41:37.076 INFO:teuthology.orchestra.run.smithi113.stdout:(18/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-04-23T01:41:37.101 INFO:teuthology.orchestra.run.smithi113.stdout:(19/32): python3-prettytable-0.7.2-27.el9.noarc 560 kB/s | 42 kB 00:00 2024-04-23T01:41:37.126 INFO:teuthology.orchestra.run.smithi113.stdout:(20/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-04-23T01:41:37.177 INFO:teuthology.orchestra.run.smithi113.stdout:(21/32): qatlib-service-24.02.0-1.el9.x86_64.rp 357 kB/s | 36 kB 00:00 2024-04-23T01:41:37.202 INFO:teuthology.orchestra.run.smithi113.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 471 kB/s | 47 kB 00:00 2024-04-23T01:41:37.278 INFO:teuthology.orchestra.run.smithi113.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 347 kB/s | 26 kB 00:00 2024-04-23T01:41:37.328 INFO:teuthology.orchestra.run.smithi113.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 976 kB/s | 49 kB 00:00 2024-04-23T01:41:37.353 INFO:teuthology.orchestra.run.smithi113.stdout:(25/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.3 MB/s | 309 kB 00:00 2024-04-23T01:41:37.378 INFO:teuthology.orchestra.run.smithi113.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-04-23T01:41:37.462 INFO:teuthology.orchestra.run.smithi113.stdout:(27/32): libarrow-9.0.0-10.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-04-23T01:41:37.487 INFO:teuthology.orchestra.run.smithi113.stdout:(28/32): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-23T01:41:37.513 INFO:teuthology.orchestra.run.smithi113.stdout:(29/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.1 MB/s | 838 kB 00:00 2024-04-23T01:41:37.555 INFO:teuthology.orchestra.run.smithi113.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-23T01:41:37.579 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-23T01:41:37.854 INFO:teuthology.orchestra.run.smithi113.stdout:(31/32): librados2-19.0.0-3192.ga1f34943.el7.x8 9.3 MB/s | 3.4 MB 00:00 2024-04-23T01:41:37.921 INFO:teuthology.orchestra.run.smithi113.stdout:(32/32): librbd1-19.0.0-3192.ga1f34943.el7.x86_ 9.0 MB/s | 3.7 MB 00:00 2024-04-23T01:41:37.923 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:41:37.923 INFO:teuthology.orchestra.run.smithi113.stdout:Total 15 MB/s | 59 MB 00:03 2024-04-23T01:41:38.019 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2024-04-23T01:41:38.055 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2024-04-23T01:41:38.055 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2024-04-23T01:41:38.181 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-04-23T01:41:38.289 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : librados2-2:19.0.0-3192.ga1f34943.el7.x86_64 2/34 2024-04-23T01:41:38.382 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados2-2:19.0.0-3192.ga1f34943.el7.x86_64 2/34 2024-04-23T01:41:38.469 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rados-2:19.0.0-3192.ga1f34943.el7.x86_64 3/34 2024-04-23T01:41:38.499 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephfs2-2:19.0.0-3192.ga1f34943.el7.x86_64 4/34 2024-04-23T01:41:38.514 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2024-04-23T01:41:38.514 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2024-04-23T01:41:38.562 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephfs2-2:19.0.0-3192.ga1f34943.el7.x86_64 4/34 2024-04-23T01:41:38.600 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : liboath-2.6.7-2.el9.x86_64 5/34 2024-04-23T01:41:38.638 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/34 2024-04-23T01:41:38.672 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/34 2024-04-23T01:41:38.702 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 8/34 2024-04-23T01:41:38.728 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 8/34 2024-04-23T01:41:38.913 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 9/34 2024-04-23T01:41:38.954 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 9/34 2024-04-23T01:41:39.312 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/34 2024-04-23T01:41:39.384 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : librbd1-2:19.0.0-3192.ga1f34943.el7.x86_64 11/34 2024-04-23T01:41:39.451 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librbd1-2:19.0.0-3192.ga1f34943.el7.x86_64 11/34 2024-04-23T01:41:39.494 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ceph-argparse-2:19.0.0-3192.ga1f34943.el7. 12/34 2024-04-23T01:41:39.549 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cephfs-2:19.0.0-3192.ga1f34943.el7.x86_64 13/34 2024-04-23T01:41:39.582 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rbd-2:19.0.0-3192.ga1f34943.el7.x86_64 14/34 2024-04-23T01:41:39.662 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 15/34 2024-04-23T01:41:39.687 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libradosstriper1-2:19.0.0-3192.ga1f34943.el7.x86_6 16/34 2024-04-23T01:41:39.775 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3192.ga1f34943.el7.x86_6 16/34 2024-04-23T01:41:39.804 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : thrift-0.15.0-2.el9.x86_64 17/34 2024-04-23T01:41:39.912 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : re2-1:20211101-3.el9.x86_64 18/34 2024-04-23T01:41:39.946 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 19/34 2024-04-23T01:41:39.972 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2024-04-23T01:41:39.986 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 20/34 2024-04-23T01:41:40.020 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 21/34 2024-04-23T01:41:40.050 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 22/34 2024-04-23T01:41:40.080 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 23/34 2024-04-23T01:41:40.117 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-04-23T01:41:40.520 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-04-23T01:41:40.590 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : librados2-2:19.0.0-3192.ga1f34943.el7.x86_64 2/34 2024-04-23T01:41:40.682 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-04-23T01:41:40.683 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librados2-2:19.0.0-3192.ga1f34943.el7.x86_64 2/34 2024-04-23T01:41:40.729 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librgw2-2:19.0.0-3192.ga1f34943.el7.x86_64 26/34 2024-04-23T01:41:40.770 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rados-2:19.0.0-3192.ga1f34943.el7.x86_64 3/34 2024-04-23T01:41:40.800 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librgw2-2:19.0.0-3192.ga1f34943.el7.x86_64 26/34 2024-04-23T01:41:40.808 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephfs2-2:19.0.0-3192.ga1f34943.el7.x86_64 4/34 2024-04-23T01:41:40.877 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rgw-2:19.0.0-3192.ga1f34943.el7.x86_64 27/34 2024-04-23T01:41:40.896 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libcephfs2-2:19.0.0-3192.ga1f34943.el7.x86_64 4/34 2024-04-23T01:41:40.907 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ceph-common-2:19.0.0-3192.ga1f34943.el7.x8 28/34 2024-04-23T01:41:40.942 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : liboath-2.6.7-2.el9.x86_64 5/34 2024-04-23T01:41:40.971 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/34 2024-04-23T01:41:40.996 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/34 2024-04-23T01:41:41.019 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 8/34 2024-04-23T01:41:41.043 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 8/34 2024-04-23T01:41:41.244 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 9/34 2024-04-23T01:41:41.277 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 9/34 2024-04-23T01:41:41.654 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/34 2024-04-23T01:41:41.698 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : librbd1-2:19.0.0-3192.ga1f34943.el7.x86_64 11/34 2024-04-23T01:41:41.783 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librbd1-2:19.0.0-3192.ga1f34943.el7.x86_64 11/34 2024-04-23T01:41:41.825 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ceph-argparse-2:19.0.0-3192.ga1f34943.el7. 12/34 2024-04-23T01:41:41.881 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cephfs-2:19.0.0-3192.ga1f34943.el7.x86_64 13/34 2024-04-23T01:41:41.905 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rbd-2:19.0.0-3192.ga1f34943.el7.x86_64 14/34 2024-04-23T01:41:41.992 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 15/34 2024-04-23T01:41:42.026 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libradosstriper1-2:19.0.0-3192.ga1f34943.el7.x86_6 16/34 2024-04-23T01:41:42.113 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3192.ga1f34943.el7.x86_6 16/34 2024-04-23T01:41:42.150 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : thrift-0.15.0-2.el9.x86_64 17/34 2024-04-23T01:41:42.260 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : re2-1:20211101-3.el9.x86_64 18/34 2024-04-23T01:41:42.292 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 19/34 2024-04-23T01:41:42.332 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 20/34 2024-04-23T01:41:42.358 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 21/34 2024-04-23T01:41:42.387 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 22/34 2024-04-23T01:41:42.418 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 23/34 2024-04-23T01:41:42.454 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-04-23T01:41:43.029 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-04-23T01:41:43.058 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librgw2-2:19.0.0-3192.ga1f34943.el7.x86_64 26/34 2024-04-23T01:41:43.129 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librgw2-2:19.0.0-3192.ga1f34943.el7.x86_64 26/34 2024-04-23T01:41:43.206 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rgw-2:19.0.0-3192.ga1f34943.el7.x86_64 27/34 2024-04-23T01:41:43.235 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ceph-common-2:19.0.0-3192.ga1f34943.el7.x8 28/34 2024-04-23T01:41:43.358 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-common-2:19.0.0-3192.ga1f34943.el7.x86_64 29/34 2024-04-23T01:41:43.389 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-common-2:19.0.0-3192.ga1f34943.el7.x86_64 29/34 2024-04-23T01:41:44.049 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-common-2:19.0.0-3192.ga1f34943.el7.x86_64 29/34 2024-04-23T01:41:44.094 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-base-2:19.0.0-3192.ga1f34943.el7.x86_64 30/34 2024-04-23T01:41:44.183 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-base-2:19.0.0-3192.ga1f34943.el7.x86_64 30/34 2024-04-23T01:41:44.183 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-23T01:41:44.183 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-23T01:41:44.183 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:41:44.212 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-selinux-2:19.0.0-3192.ga1f34943.el7.x86_64 31/34 2024-04-23T01:41:45.703 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-common-2:19.0.0-3192.ga1f34943.el7.x86_64 29/34 2024-04-23T01:41:45.940 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-common-2:19.0.0-3192.ga1f34943.el7.x86_64 29/34 2024-04-23T01:41:46.607 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-common-2:19.0.0-3192.ga1f34943.el7.x86_64 29/34 2024-04-23T01:41:46.651 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-base-2:19.0.0-3192.ga1f34943.el7.x86_64 30/34 2024-04-23T01:41:46.742 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-base-2:19.0.0-3192.ga1f34943.el7.x86_64 30/34 2024-04-23T01:41:46.742 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-23T01:41:46.742 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-23T01:41:46.742 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:41:46.769 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-selinux-2:19.0.0-3192.ga1f34943.el7.x86_64 31/34 2024-04-23T01:41:54.856 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3192.ga1f34943.el7.x86_64 31/34 2024-04-23T01:41:54.856 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /sys 2024-04-23T01:41:54.856 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /proc 2024-04-23T01:41:54.856 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /mnt 2024-04-23T01:41:54.856 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /var/tmp 2024-04-23T01:41:54.856 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /home 2024-04-23T01:41:54.856 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /root 2024-04-23T01:41:54.856 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /tmp 2024-04-23T01:41:54.856 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:41:55.923 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-radosgw-2:19.0.0-3192.ga1f34943.el7.x86_64 32/34 2024-04-23T01:41:55.947 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3192.ga1f34943.el7.x86_64 32/34 2024-04-23T01:41:55.947 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T01:41:55.947 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-23T01:41:55.947 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T01:41:55.947 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T01:41:55.947 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:41:55.949 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-23T01:41:56.034 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-23T01:41:56.034 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-23T01:41:56.751 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3192.ga1f34943.el7.x86_64 31/34 2024-04-23T01:41:56.751 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /sys 2024-04-23T01:41:56.751 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /proc 2024-04-23T01:41:56.751 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /mnt 2024-04-23T01:41:56.751 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /var/tmp 2024-04-23T01:41:56.751 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /home 2024-04-23T01:41:56.751 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /root 2024-04-23T01:41:56.751 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /tmp 2024-04-23T01:41:56.752 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:41:57.567 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-23T01:41:57.567 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-base-2:19.0.0-3192.ga1f34943.el7.x86_64 1/34 2024-04-23T01:41:57.567 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-common-2:19.0.0-3192.ga1f34943.el7.x86_64 2/34 2024-04-23T01:41:57.567 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-radosgw-2:19.0.0-3192.ga1f34943.el7.x86_64 3/34 2024-04-23T01:41:57.567 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-selinux-2:19.0.0-3192.ga1f34943.el7.x86_64 4/34 2024-04-23T01:41:57.567 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephfs2-2:19.0.0-3192.ga1f34943.el7.x86_64 5/34 2024-04-23T01:41:57.568 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libradosstriper1-2:19.0.0-3192.ga1f34943.el7.x86_6 6/34 2024-04-23T01:41:57.568 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librgw2-2:19.0.0-3192.ga1f34943.el7.x86_64 7/34 2024-04-23T01:41:57.568 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3192.ga1f34943.el7. 8/34 2024-04-23T01:41:57.568 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ceph-common-2:19.0.0-3192.ga1f34943.el7.x8 9/34 2024-04-23T01:41:57.568 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cephfs-2:19.0.0-3192.ga1f34943.el7.x86_64 10/34 2024-04-23T01:41:57.568 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rados-2:19.0.0-3192.ga1f34943.el7.x86_64 11/34 2024-04-23T01:41:57.568 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rbd-2:19.0.0-3192.ga1f34943.el7.x86_64 12/34 2024-04-23T01:41:57.568 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rgw-2:19.0.0-3192.ga1f34943.el7.x86_64 13/34 2024-04-23T01:41:57.568 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-04-23T01:41:57.569 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-04-23T01:41:57.569 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-04-23T01:41:57.569 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-04-23T01:41:57.569 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-04-23T01:41:57.569 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-04-23T01:41:57.569 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-04-23T01:41:57.569 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-04-23T01:41:57.569 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-04-23T01:41:57.570 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-04-23T01:41:57.570 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-04-23T01:41:57.570 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-04-23T01:41:57.570 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-04-23T01:41:57.570 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-04-23T01:41:57.570 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-04-23T01:41:57.570 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-04-23T01:41:57.570 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-04-23T01:41:57.570 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados2-2:19.0.0-3192.ga1f34943.el7.x86_64 31/34 2024-04-23T01:41:57.570 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-04-23T01:41:57.571 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librbd1-2:19.0.0-3192.ga1f34943.el7.x86_64 33/34 2024-04-23T01:41:57.822 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-radosgw-2:19.0.0-3192.ga1f34943.el7.x86_64 32/34 2024-04-23T01:41:57.844 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3192.ga1f34943.el7.x86_64 32/34 2024-04-23T01:41:57.844 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T01:41:57.845 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-23T01:41:57.845 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T01:41:57.845 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T01:41:57.845 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:41:57.846 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-23T01:41:57.939 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-23T01:41:57.939 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-23T01:41:58.373 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-04-23T01:41:58.374 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:41:58.374 INFO:teuthology.orchestra.run.smithi090.stdout:Upgraded: 2024-04-23T01:41:58.374 INFO:teuthology.orchestra.run.smithi090.stdout: librados2-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:41:58.374 INFO:teuthology.orchestra.run.smithi090.stdout: librbd1-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:41:58.374 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-23T01:41:58.374 INFO:teuthology.orchestra.run.smithi090.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-23T01:41:58.374 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:41:58.374 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-common-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:41:58.374 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-radosgw-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:41:58.374 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-selinux-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:41:58.374 INFO:teuthology.orchestra.run.smithi090.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-23T01:41:58.374 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-23T01:41:58.374 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-23T01:41:58.375 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:41:58.375 INFO:teuthology.orchestra.run.smithi090.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-23T01:41:58.375 INFO:teuthology.orchestra.run.smithi090.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-23T01:41:58.375 INFO:teuthology.orchestra.run.smithi090.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-23T01:41:58.375 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:41:58.375 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-23T01:41:58.375 INFO:teuthology.orchestra.run.smithi090.stdout: librgw2-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:41:58.375 INFO:teuthology.orchestra.run.smithi090.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-23T01:41:58.375 INFO:teuthology.orchestra.run.smithi090.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-23T01:41:58.375 INFO:teuthology.orchestra.run.smithi090.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-23T01:41:58.375 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:41:58.375 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-common-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:41:58.375 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cephfs-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:41:58.376 INFO:teuthology.orchestra.run.smithi090.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-23T01:41:58.376 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:41:58.376 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rbd-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:41:58.376 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rgw-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:41:58.376 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-23T01:41:58.376 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-23T01:41:58.376 INFO:teuthology.orchestra.run.smithi090.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-23T01:41:58.376 INFO:teuthology.orchestra.run.smithi090.stdout: re2-1:20211101-3.el9.x86_64 2024-04-23T01:41:58.376 INFO:teuthology.orchestra.run.smithi090.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-23T01:41:58.376 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:41:58.376 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-23T01:41:58.659 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-test 2024-04-23T01:41:59.196 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:00:32 ago on Tue 23 Apr 2024 01:41:27 AM UTC. 2024-04-23T01:41:59.317 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-23T01:41:59.318 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-base-2:19.0.0-3192.ga1f34943.el7.x86_64 1/34 2024-04-23T01:41:59.318 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-common-2:19.0.0-3192.ga1f34943.el7.x86_64 2/34 2024-04-23T01:41:59.318 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-radosgw-2:19.0.0-3192.ga1f34943.el7.x86_64 3/34 2024-04-23T01:41:59.318 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-selinux-2:19.0.0-3192.ga1f34943.el7.x86_64 4/34 2024-04-23T01:41:59.318 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephfs2-2:19.0.0-3192.ga1f34943.el7.x86_64 5/34 2024-04-23T01:41:59.318 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libradosstriper1-2:19.0.0-3192.ga1f34943.el7.x86_6 6/34 2024-04-23T01:41:59.318 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librgw2-2:19.0.0-3192.ga1f34943.el7.x86_64 7/34 2024-04-23T01:41:59.318 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3192.ga1f34943.el7. 8/34 2024-04-23T01:41:59.318 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ceph-common-2:19.0.0-3192.ga1f34943.el7.x8 9/34 2024-04-23T01:41:59.318 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cephfs-2:19.0.0-3192.ga1f34943.el7.x86_64 10/34 2024-04-23T01:41:59.318 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rados-2:19.0.0-3192.ga1f34943.el7.x86_64 11/34 2024-04-23T01:41:59.318 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rbd-2:19.0.0-3192.ga1f34943.el7.x86_64 12/34 2024-04-23T01:41:59.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rgw-2:19.0.0-3192.ga1f34943.el7.x86_64 13/34 2024-04-23T01:41:59.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-04-23T01:41:59.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-04-23T01:41:59.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-04-23T01:41:59.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-04-23T01:41:59.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-04-23T01:41:59.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-04-23T01:41:59.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-04-23T01:41:59.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-04-23T01:41:59.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-04-23T01:41:59.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-04-23T01:41:59.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-04-23T01:41:59.320 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-04-23T01:41:59.320 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-04-23T01:41:59.320 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-04-23T01:41:59.320 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-04-23T01:41:59.320 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-04-23T01:41:59.321 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-04-23T01:41:59.321 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados2-2:19.0.0-3192.ga1f34943.el7.x86_64 31/34 2024-04-23T01:41:59.321 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-04-23T01:41:59.321 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librbd1-2:19.0.0-3192.ga1f34943.el7.x86_64 33/34 2024-04-23T01:41:59.322 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-23T01:41:59.322 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:41:59.322 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-04-23T01:41:59.322 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:41:59.322 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-23T01:41:59.322 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-test x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 84 M 2024-04-23T01:41:59.322 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-04-23T01:41:59.322 INFO:teuthology.orchestra.run.smithi090.stdout: libasan8 x86_64 13.2.1-6.3.el9 appstream 469 k 2024-04-23T01:41:59.322 INFO:teuthology.orchestra.run.smithi090.stdout: libcephsqlite x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 173 k 2024-04-23T01:41:59.322 INFO:teuthology.orchestra.run.smithi090.stdout: libubsan x86_64 11.4.1-3.el9 appstream 187 k 2024-04-23T01:41:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2024-04-23T01:41:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-23T01:41:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-23T01:41:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-23T01:41:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2024-04-23T01:41:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:41:59.323 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-23T01:41:59.323 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:41:59.323 INFO:teuthology.orchestra.run.smithi090.stdout:Install 9 Packages 2024-04-23T01:41:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:41:59.323 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 87 M 2024-04-23T01:41:59.323 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 813 M 2024-04-23T01:41:59.323 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-23T01:42:00.091 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-04-23T01:42:00.091 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:42:00.091 INFO:teuthology.orchestra.run.smithi113.stdout:Upgraded: 2024-04-23T01:42:00.091 INFO:teuthology.orchestra.run.smithi113.stdout: librados2-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:42:00.091 INFO:teuthology.orchestra.run.smithi113.stdout: librbd1-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:42:00.092 INFO:teuthology.orchestra.run.smithi113.stdout:Installed: 2024-04-23T01:42:00.092 INFO:teuthology.orchestra.run.smithi113.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-23T01:42:00.092 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-base-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:42:00.092 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-common-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:42:00.092 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-radosgw-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:42:00.092 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-selinux-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:42:00.092 INFO:teuthology.orchestra.run.smithi113.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-23T01:42:00.092 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-23T01:42:00.092 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-23T01:42:00.092 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs2-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:42:00.092 INFO:teuthology.orchestra.run.smithi113.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-23T01:42:00.092 INFO:teuthology.orchestra.run.smithi113.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-23T01:42:00.093 INFO:teuthology.orchestra.run.smithi113.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-23T01:42:00.093 INFO:teuthology.orchestra.run.smithi113.stdout: libradosstriper1-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:42:00.093 INFO:teuthology.orchestra.run.smithi113.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-23T01:42:00.093 INFO:teuthology.orchestra.run.smithi113.stdout: librgw2-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:42:00.093 INFO:teuthology.orchestra.run.smithi113.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-23T01:42:00.093 INFO:teuthology.orchestra.run.smithi113.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-23T01:42:00.093 INFO:teuthology.orchestra.run.smithi113.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-23T01:42:00.093 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-argparse-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:42:00.093 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-common-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:42:00.093 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cephfs-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:42:00.093 INFO:teuthology.orchestra.run.smithi113.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-23T01:42:00.093 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rados-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:42:00.093 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rbd-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:42:00.094 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rgw-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:42:00.094 INFO:teuthology.orchestra.run.smithi113.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-23T01:42:00.094 INFO:teuthology.orchestra.run.smithi113.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-23T01:42:00.094 INFO:teuthology.orchestra.run.smithi113.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-23T01:42:00.094 INFO:teuthology.orchestra.run.smithi113.stdout: re2-1:20211101-3.el9.x86_64 2024-04-23T01:42:00.094 INFO:teuthology.orchestra.run.smithi113.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-23T01:42:00.094 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:42:00.094 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-23T01:42:00.297 INFO:teuthology.orchestra.run.smithi090.stdout:(1/9): libcephsqlite-19.0.0-3192.ga1f34943.el7. 316 kB/s | 173 kB 00:00 2024-04-23T01:42:00.389 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install ceph-test 2024-04-23T01:42:00.473 INFO:teuthology.orchestra.run.smithi090.stdout:(2/9): libasan8-13.2.1-6.3.el9.x86_64.rpm 2.6 MB/s | 469 kB 00:00 2024-04-23T01:42:00.532 INFO:teuthology.orchestra.run.smithi090.stdout:(3/9): lksctp-tools-1.0.19-2.el9.x86_64.rpm 120 kB/s | 94 kB 00:00 2024-04-23T01:42:00.583 INFO:teuthology.orchestra.run.smithi090.stdout:(4/9): libubsan-11.4.1-3.el9.x86_64.rpm 1.7 MB/s | 187 kB 00:00 2024-04-23T01:42:00.660 INFO:teuthology.orchestra.run.smithi090.stdout:(5/9): socat-1.7.4.1-5.el9.x86_64.rpm 3.9 MB/s | 305 kB 00:00 2024-04-23T01:42:00.727 INFO:teuthology.orchestra.run.smithi090.stdout:(6/9): xmlstarlet-1.6.1-20.el9.x86_64.rpm 953 kB/s | 64 kB 00:00 2024-04-23T01:42:00.820 INFO:teuthology.orchestra.run.smithi090.stdout:(7/9): protobuf-3.14.0-13.el9.x86_64.rpm 3.5 MB/s | 1.0 MB 00:00 2024-04-23T01:42:00.887 INFO:teuthology.orchestra.run.smithi090.stdout:(8/9): yaml-cpp-0.6.3-5.el9.x86_64.rpm 763 kB/s | 122 kB 00:00 2024-04-23T01:42:00.967 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:00:29 ago on Tue 23 Apr 2024 01:41:31 AM UTC. 2024-04-23T01:42:01.094 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-23T01:42:01.095 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:42:01.095 INFO:teuthology.orchestra.run.smithi113.stdout: Package Arch Version Repository Size 2024-04-23T01:42:01.095 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:42:01.095 INFO:teuthology.orchestra.run.smithi113.stdout:Installing: 2024-04-23T01:42:01.095 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-test x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 84 M 2024-04-23T01:42:01.095 INFO:teuthology.orchestra.run.smithi113.stdout:Installing dependencies: 2024-04-23T01:42:01.095 INFO:teuthology.orchestra.run.smithi113.stdout: libasan8 x86_64 13.2.1-6.3.el9 appstream 469 k 2024-04-23T01:42:01.096 INFO:teuthology.orchestra.run.smithi113.stdout: libcephsqlite x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 173 k 2024-04-23T01:42:01.096 INFO:teuthology.orchestra.run.smithi113.stdout: libubsan x86_64 11.4.1-3.el9 appstream 187 k 2024-04-23T01:42:01.096 INFO:teuthology.orchestra.run.smithi113.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2024-04-23T01:42:01.096 INFO:teuthology.orchestra.run.smithi113.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-23T01:42:01.096 INFO:teuthology.orchestra.run.smithi113.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-23T01:42:01.096 INFO:teuthology.orchestra.run.smithi113.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-23T01:42:01.096 INFO:teuthology.orchestra.run.smithi113.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2024-04-23T01:42:01.096 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:42:01.096 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-04-23T01:42:01.096 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:42:01.096 INFO:teuthology.orchestra.run.smithi113.stdout:Install 9 Packages 2024-04-23T01:42:01.096 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:42:01.097 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 87 M 2024-04-23T01:42:01.097 INFO:teuthology.orchestra.run.smithi113.stdout:Installed size: 813 M 2024-04-23T01:42:01.097 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2024-04-23T01:42:02.187 INFO:teuthology.orchestra.run.smithi113.stdout:(1/9): lksctp-tools-1.0.19-2.el9.x86_64.rpm 513 kB/s | 94 kB 00:00 2024-04-23T01:42:02.322 INFO:teuthology.orchestra.run.smithi113.stdout:(2/9): libasan8-13.2.1-6.3.el9.x86_64.rpm 3.4 MB/s | 469 kB 00:00 2024-04-23T01:42:02.398 INFO:teuthology.orchestra.run.smithi113.stdout:(3/9): libubsan-11.4.1-3.el9.x86_64.rpm 2.4 MB/s | 187 kB 00:00 2024-04-23T01:42:02.533 INFO:teuthology.orchestra.run.smithi113.stdout:(4/9): protobuf-3.14.0-13.el9.x86_64.rpm 7.5 MB/s | 1.0 MB 00:00 2024-04-23T01:42:02.569 INFO:teuthology.orchestra.run.smithi113.stdout:(5/9): libcephsqlite-19.0.0-3192.ga1f34943.el7. 313 kB/s | 173 kB 00:00 2024-04-23T01:42:02.616 INFO:teuthology.orchestra.run.smithi113.stdout:(6/9): socat-1.7.4.1-5.el9.x86_64.rpm 3.6 MB/s | 305 kB 00:00 2024-04-23T01:42:02.726 INFO:teuthology.orchestra.run.smithi113.stdout:(7/9): xmlstarlet-1.6.1-20.el9.x86_64.rpm 379 kB/s | 64 kB 00:00 2024-04-23T01:42:03.052 INFO:teuthology.orchestra.run.smithi113.stdout:(8/9): yaml-cpp-0.6.3-5.el9.x86_64.rpm 280 kB/s | 122 kB 00:00 2024-04-23T01:42:07.293 INFO:teuthology.orchestra.run.smithi113.stdout:(9/9): ceph-test-19.0.0-3192.ga1f34943.el7.x86_ 16 MB/s | 84 MB 00:05 2024-04-23T01:42:07.296 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:42:07.296 INFO:teuthology.orchestra.run.smithi113.stdout:Total 14 MB/s | 87 MB 00:06 2024-04-23T01:42:07.376 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2024-04-23T01:42:07.399 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2024-04-23T01:42:07.399 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2024-04-23T01:42:08.003 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2024-04-23T01:42:08.003 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2024-04-23T01:42:08.884 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2024-04-23T01:42:08.958 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/9 2024-04-23T01:42:09.002 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 2/9 2024-04-23T01:42:09.047 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 3/9 2024-04-23T01:42:09.083 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 4/9 2024-04-23T01:42:09.112 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libubsan-11.4.1-3.el9.x86_64 5/9 2024-04-23T01:42:09.138 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libasan8-13.2.1-6.3.el9.x86_64 6/9 2024-04-23T01:42:09.207 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libasan8-13.2.1-6.3.el9.x86_64 6/9 2024-04-23T01:42:09.246 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 7/9 2024-04-23T01:42:09.273 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephsqlite-2:19.0.0-3192.ga1f34943.el7.x86_64 8/9 2024-04-23T01:42:14.507 INFO:teuthology.orchestra.run.smithi090.stdout:(9/9): ceph-test-19.0.0-3192.ga1f34943.el7.x86_ 5.7 MB/s | 84 MB 00:14 2024-04-23T01:42:14.508 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:42:14.508 INFO:teuthology.orchestra.run.smithi090.stdout:Total 5.7 MB/s | 87 MB 00:15 2024-04-23T01:42:14.590 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-23T01:42:14.613 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-23T01:42:14.613 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-23T01:42:15.214 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-23T01:42:15.215 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-23T01:42:16.087 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-23T01:42:16.181 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/9 2024-04-23T01:42:16.234 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 2/9 2024-04-23T01:42:16.288 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 3/9 2024-04-23T01:42:16.323 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 4/9 2024-04-23T01:42:16.361 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libubsan-11.4.1-3.el9.x86_64 5/9 2024-04-23T01:42:16.387 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libasan8-13.2.1-6.3.el9.x86_64 6/9 2024-04-23T01:42:16.447 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libasan8-13.2.1-6.3.el9.x86_64 6/9 2024-04-23T01:42:16.488 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 7/9 2024-04-23T01:42:16.522 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephsqlite-2:19.0.0-3192.ga1f34943.el7.x86_64 8/9 2024-04-23T01:42:20.589 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3192.ga1f34943.el7.x86_64 8/9 2024-04-23T01:42:20.944 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-test-2:19.0.0-3192.ga1f34943.el7.x86_64 9/9 2024-04-23T01:42:27.154 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-test-2:19.0.0-3192.ga1f34943.el7.x86_64 9/9 2024-04-23T01:42:27.154 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-test-2:19.0.0-3192.ga1f34943.el7.x86_64 1/9 2024-04-23T01:42:27.154 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephsqlite-2:19.0.0-3192.ga1f34943.el7.x86_64 2/9 2024-04-23T01:42:27.154 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 3/9 2024-04-23T01:42:27.154 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libasan8-13.2.1-6.3.el9.x86_64 4/9 2024-04-23T01:42:27.154 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libubsan-11.4.1-3.el9.x86_64 5/9 2024-04-23T01:42:27.154 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 6/9 2024-04-23T01:42:27.154 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 7/9 2024-04-23T01:42:27.154 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 8/9 2024-04-23T01:42:27.485 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 9/9 2024-04-23T01:42:27.485 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:42:27.485 INFO:teuthology.orchestra.run.smithi113.stdout:Installed: 2024-04-23T01:42:27.485 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-test-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:42:27.485 INFO:teuthology.orchestra.run.smithi113.stdout: libasan8-13.2.1-6.3.el9.x86_64 2024-04-23T01:42:27.485 INFO:teuthology.orchestra.run.smithi113.stdout: libcephsqlite-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:42:27.485 INFO:teuthology.orchestra.run.smithi113.stdout: libubsan-11.4.1-3.el9.x86_64 2024-04-23T01:42:27.485 INFO:teuthology.orchestra.run.smithi113.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2024-04-23T01:42:27.485 INFO:teuthology.orchestra.run.smithi113.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-23T01:42:27.486 INFO:teuthology.orchestra.run.smithi113.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-23T01:42:27.486 INFO:teuthology.orchestra.run.smithi113.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-23T01:42:27.486 INFO:teuthology.orchestra.run.smithi113.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2024-04-23T01:42:27.486 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:42:27.486 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-23T01:42:27.798 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install ceph 2024-04-23T01:42:27.823 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3192.ga1f34943.el7.x86_64 8/9 2024-04-23T01:42:27.905 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-test-2:19.0.0-3192.ga1f34943.el7.x86_64 9/9 2024-04-23T01:42:28.311 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:00:57 ago on Tue 23 Apr 2024 01:41:31 AM UTC. 2024-04-23T01:42:28.431 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-23T01:42:28.434 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:42:28.434 INFO:teuthology.orchestra.run.smithi113.stdout: Package Arch Version Repository Size 2024-04-23T01:42:28.434 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:42:28.434 INFO:teuthology.orchestra.run.smithi113.stdout:Installing: 2024-04-23T01:42:28.434 INFO:teuthology.orchestra.run.smithi113.stdout: ceph x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 6.5 k 2024-04-23T01:42:28.434 INFO:teuthology.orchestra.run.smithi113.stdout:Installing dependencies: 2024-04-23T01:42:28.434 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mds x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 2.2 M 2024-04-23T01:42:28.435 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 1.6 M 2024-04-23T01:42:28.435 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3192.ga1f34943.el7 ceph-noarch 249 k 2024-04-23T01:42:28.435 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mon x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 5.0 M 2024-04-23T01:42:28.435 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-osd x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 71 M 2024-04-23T01:42:28.435 INFO:teuthology.orchestra.run.smithi113.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-23T01:42:28.435 INFO:teuthology.orchestra.run.smithi113.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-23T01:42:28.435 INFO:teuthology.orchestra.run.smithi113.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-23T01:42:28.435 INFO:teuthology.orchestra.run.smithi113.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-23T01:42:28.435 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-23T01:42:28.435 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-23T01:42:28.435 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-23T01:42:28.435 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-23T01:42:28.435 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-23T01:42:28.436 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-23T01:42:28.436 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-collections 2024-04-23T01:42:28.436 INFO:teuthology.orchestra.run.smithi113.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-04-23T01:42:28.436 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-23T01:42:28.436 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-23T01:42:28.436 INFO:teuthology.orchestra.run.smithi113.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-23T01:42:28.436 INFO:teuthology.orchestra.run.smithi113.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-23T01:42:28.436 INFO:teuthology.orchestra.run.smithi113.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-23T01:42:28.436 INFO:teuthology.orchestra.run.smithi113.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-23T01:42:28.436 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-23T01:42:28.436 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-23T01:42:28.437 INFO:teuthology.orchestra.run.smithi113.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-23T01:42:28.437 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-23T01:42:28.437 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-23T01:42:28.437 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-23T01:42:28.437 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-23T01:42:28.437 INFO:teuthology.orchestra.run.smithi113.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-23T01:42:28.437 INFO:teuthology.orchestra.run.smithi113.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-23T01:42:28.437 INFO:teuthology.orchestra.run.smithi113.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-23T01:42:28.437 INFO:teuthology.orchestra.run.smithi113.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-23T01:42:28.437 INFO:teuthology.orchestra.run.smithi113.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-23T01:42:28.437 INFO:teuthology.orchestra.run.smithi113.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-23T01:42:28.437 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:42:28.437 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-04-23T01:42:28.437 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:42:28.438 INFO:teuthology.orchestra.run.smithi113.stdout:Install 36 Packages 2024-04-23T01:42:28.438 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:42:28.438 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 85 M 2024-04-23T01:42:28.438 INFO:teuthology.orchestra.run.smithi113.stdout:Installed size: 923 M 2024-04-23T01:42:28.438 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2024-04-23T01:42:29.297 INFO:teuthology.orchestra.run.smithi113.stdout:(1/36): ceph-19.0.0-3192.ga1f34943.el7.x86_64.r 28 kB/s | 6.5 kB 00:00 2024-04-23T01:42:29.415 INFO:teuthology.orchestra.run.smithi113.stdout:(2/36): ceph-mgr-19.0.0-3192.ga1f34943.el7.x86_ 4.4 MB/s | 1.6 MB 00:00 2024-04-23T01:42:30.564 INFO:teuthology.orchestra.run.smithi113.stdout:(3/36): ceph-mds-19.0.0-3192.ga1f34943.el7.x86_ 1.5 MB/s | 2.2 MB 00:01 2024-04-23T01:42:30.748 INFO:teuthology.orchestra.run.smithi113.stdout:(4/36): ceph-mgr-modules-core-19.0.0-3192.ga1f3 1.3 MB/s | 249 kB 00:00 2024-04-23T01:42:31.481 INFO:teuthology.orchestra.run.smithi113.stdout:(5/36): ceph-mon-19.0.0-3192.ga1f34943.el7.x86_ 2.3 MB/s | 5.0 MB 00:02 2024-04-23T01:42:31.798 INFO:teuthology.orchestra.run.smithi113.stdout:(6/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 241 kB/s | 253 kB 00:01 2024-04-23T01:42:32.799 INFO:teuthology.orchestra.run.smithi113.stdout:(7/36): ceph-osd-19.0.0-3192.ga1f34943.el7.x86_ 21 MB/s | 71 MB 00:03 2024-04-23T01:42:33.165 INFO:teuthology.orchestra.run.smithi113.stdout:(8/36): python3-ply-3.11-14.el9.noarch.rpm 78 kB/s | 106 kB 00:01 2024-04-23T01:42:33.591 INFO:teuthology.orchestra.run.smithi113.stdout:(9/36): python3-pycparser-2.20-6.el9.noarch.rpm 171 kB/s | 135 kB 00:00 2024-04-23T01:42:33.991 INFO:teuthology.orchestra.run.smithi113.stdout:(10/36): python3-requests-2.25.1-8.el9.noarch.r 151 kB/s | 125 kB 00:00 2024-04-23T01:42:34.250 INFO:teuthology.orchestra.run.smithi113.stdout:(11/36): python3-cryptography-36.0.1-4.el9.x86_ 453 kB/s | 1.2 MB 00:02 2024-04-23T01:42:34.291 INFO:teuthology.orchestra.run.smithi113.stdout:(12/36): lua-5.4.4-4.el9.x86_64.rpm 629 kB/s | 188 kB 00:00 2024-04-23T01:42:34.383 INFO:teuthology.orchestra.run.smithi113.stdout:(13/36): python3-markupsafe-1.1.1-12.el9.x86_64 379 kB/s | 35 kB 00:00 2024-04-23T01:42:34.450 INFO:teuthology.orchestra.run.smithi113.stdout:(14/36): python3-pytz-2021.1-5.el9.noarch.rpm 767 kB/s | 51 kB 00:00 2024-04-23T01:42:34.501 INFO:teuthology.orchestra.run.smithi113.stdout:(15/36): python3-mako-1.1.4-6.el9.noarch.rpm 688 kB/s | 172 kB 00:00 2024-04-23T01:42:34.526 INFO:teuthology.orchestra.run.smithi113.stdout:(16/36): python3-toml-0.10.2-6.el9.noarch.rpm 555 kB/s | 42 kB 00:00 2024-04-23T01:42:34.568 INFO:teuthology.orchestra.run.smithi113.stdout:(17/36): python3-urllib3-1.26.5-5.el9.noarch.rp 220 kB/s | 215 kB 00:00 2024-04-23T01:42:34.668 INFO:teuthology.orchestra.run.smithi113.stdout:(18/36): lua-devel-5.4.4-4.el9.x86_64.rpm 133 kB/s | 22 kB 00:00 2024-04-23T01:42:34.801 INFO:teuthology.orchestra.run.smithi113.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 210 kB/s | 43 kB 00:00 2024-04-23T01:42:34.827 INFO:teuthology.orchestra.run.smithi113.stdout:(20/36): luarocks-3.9.2-1.el9.noarch.rpm 503 kB/s | 151 kB 00:00 2024-04-23T01:42:34.869 INFO:teuthology.orchestra.run.smithi113.stdout:(21/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 257 kB/s | 11 kB 00:00 2024-04-23T01:42:34.911 INFO:teuthology.orchestra.run.smithi113.stdout:(22/36): python3-jaraco-classes-3.2.1-5.el9.noa 423 kB/s | 18 kB 00:00 2024-04-23T01:42:34.953 INFO:teuthology.orchestra.run.smithi113.stdout:(23/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 606 kB/s | 172 kB 00:00 2024-04-23T01:42:34.978 INFO:teuthology.orchestra.run.smithi113.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 346 kB/s | 23 kB 00:00 2024-04-23T01:42:35.003 INFO:teuthology.orchestra.run.smithi113.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 386 kB/s | 19 kB 00:00 2024-04-23T01:42:35.028 INFO:teuthology.orchestra.run.smithi113.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 393 kB/s | 20 kB 00:00 2024-04-23T01:42:35.054 INFO:teuthology.orchestra.run.smithi113.stdout:(27/36): python3-cherrypy-18.6.1-2.el9.noarch.r 1.4 MB/s | 358 kB 00:00 2024-04-23T01:42:35.079 INFO:teuthology.orchestra.run.smithi113.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 615 kB/s | 46 kB 00:00 2024-04-23T01:42:35.096 INFO:teuthology.orchestra.run.smithi113.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 1.1 MB/s | 79 kB 00:00 2024-04-23T01:42:35.121 INFO:teuthology.orchestra.run.smithi113.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 394 kB/s | 16 kB 00:00 2024-04-23T01:42:35.163 INFO:teuthology.orchestra.run.smithi113.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.3 MB/s | 90 kB 00:00 2024-04-23T01:42:35.189 INFO:teuthology.orchestra.run.smithi113.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 534 kB/s | 36 kB 00:00 2024-04-23T01:42:35.231 INFO:teuthology.orchestra.run.smithi113.stdout:(33/36): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-04-23T01:42:35.272 INFO:teuthology.orchestra.run.smithi113.stdout:(34/36): python3-zc-lockfile-2.0-10.el9.noarch. 482 kB/s | 20 kB 00:00 2024-04-23T01:42:35.306 INFO:teuthology.orchestra.run.smithi113.stdout:(35/36): python3-webob-1.8.7-6.el9.noarch.rpm 1.6 MB/s | 230 kB 00:00 2024-04-23T01:42:35.423 INFO:teuthology.orchestra.run.smithi113.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.8 MB/s | 427 kB 00:00 2024-04-23T01:42:35.427 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:42:35.428 INFO:teuthology.orchestra.run.smithi113.stdout:Total 12 MB/s | 85 MB 00:06 2024-04-23T01:42:35.563 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2024-04-23T01:42:35.605 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2024-04-23T01:42:35.605 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2024-04-23T01:42:35.771 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-test-2:19.0.0-3192.ga1f34943.el7.x86_64 9/9 2024-04-23T01:42:35.771 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-test-2:19.0.0-3192.ga1f34943.el7.x86_64 1/9 2024-04-23T01:42:35.771 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephsqlite-2:19.0.0-3192.ga1f34943.el7.x86_64 2/9 2024-04-23T01:42:35.771 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 3/9 2024-04-23T01:42:35.771 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libasan8-13.2.1-6.3.el9.x86_64 4/9 2024-04-23T01:42:35.771 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libubsan-11.4.1-3.el9.x86_64 5/9 2024-04-23T01:42:35.771 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 6/9 2024-04-23T01:42:35.771 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 7/9 2024-04-23T01:42:35.771 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 8/9 2024-04-23T01:42:36.157 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 9/9 2024-04-23T01:42:36.158 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:42:36.158 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-23T01:42:36.158 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-test-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:42:36.158 INFO:teuthology.orchestra.run.smithi090.stdout: libasan8-13.2.1-6.3.el9.x86_64 2024-04-23T01:42:36.158 INFO:teuthology.orchestra.run.smithi090.stdout: libcephsqlite-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:42:36.158 INFO:teuthology.orchestra.run.smithi090.stdout: libubsan-11.4.1-3.el9.x86_64 2024-04-23T01:42:36.158 INFO:teuthology.orchestra.run.smithi090.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2024-04-23T01:42:36.158 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-23T01:42:36.158 INFO:teuthology.orchestra.run.smithi090.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-23T01:42:36.158 INFO:teuthology.orchestra.run.smithi090.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-23T01:42:36.158 INFO:teuthology.orchestra.run.smithi090.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2024-04-23T01:42:36.158 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:42:36.158 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-23T01:42:36.215 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2024-04-23T01:42:36.215 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2024-04-23T01:42:36.486 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph 2024-04-23T01:42:37.005 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:10 ago on Tue 23 Apr 2024 01:41:27 AM UTC. 2024-04-23T01:42:37.126 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-23T01:42:37.129 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:42:37.129 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-04-23T01:42:37.129 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:42:37.129 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-23T01:42:37.129 INFO:teuthology.orchestra.run.smithi090.stdout: ceph x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 6.5 k 2024-04-23T01:42:37.129 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-04-23T01:42:37.129 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 2.2 M 2024-04-23T01:42:37.130 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 1.6 M 2024-04-23T01:42:37.130 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3192.ga1f34943.el7 ceph-noarch 249 k 2024-04-23T01:42:37.130 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 5.0 M 2024-04-23T01:42:37.130 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-osd x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 71 M 2024-04-23T01:42:37.130 INFO:teuthology.orchestra.run.smithi090.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-23T01:42:37.130 INFO:teuthology.orchestra.run.smithi090.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-23T01:42:37.130 INFO:teuthology.orchestra.run.smithi090.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-23T01:42:37.130 INFO:teuthology.orchestra.run.smithi090.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-23T01:42:37.130 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-23T01:42:37.130 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-23T01:42:37.130 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-23T01:42:37.130 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-23T01:42:37.130 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-23T01:42:37.131 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-23T01:42:37.131 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-collections 2024-04-23T01:42:37.131 INFO:teuthology.orchestra.run.smithi090.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-04-23T01:42:37.131 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-23T01:42:37.131 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-23T01:42:37.131 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-23T01:42:37.131 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-23T01:42:37.131 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-23T01:42:37.131 INFO:teuthology.orchestra.run.smithi090.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-23T01:42:37.131 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-23T01:42:37.131 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-23T01:42:37.131 INFO:teuthology.orchestra.run.smithi090.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-23T01:42:37.131 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-23T01:42:37.131 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-23T01:42:37.131 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-23T01:42:37.132 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-23T01:42:37.132 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-23T01:42:37.132 INFO:teuthology.orchestra.run.smithi090.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-23T01:42:37.132 INFO:teuthology.orchestra.run.smithi090.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-23T01:42:37.132 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-23T01:42:37.132 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-23T01:42:37.132 INFO:teuthology.orchestra.run.smithi090.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-23T01:42:37.132 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:42:37.132 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-23T01:42:37.132 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:42:37.132 INFO:teuthology.orchestra.run.smithi090.stdout:Install 36 Packages 2024-04-23T01:42:37.132 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:42:37.133 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 85 M 2024-04-23T01:42:37.133 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 923 M 2024-04-23T01:42:37.133 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-23T01:42:37.790 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2024-04-23T01:42:37.858 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-04-23T01:42:37.891 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-04-23T01:42:37.919 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-04-23T01:42:37.990 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-04-23T01:42:38.015 INFO:teuthology.orchestra.run.smithi090.stdout:(1/36): ceph-19.0.0-3192.ga1f34943.el7.x86_64.r 21 kB/s | 6.5 kB 00:00 2024-04-23T01:42:38.051 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-04-23T01:42:38.084 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-04-23T01:42:38.106 INFO:teuthology.orchestra.run.smithi090.stdout:(2/36): ceph-mgr-19.0.0-3192.ga1f34943.el7.x86_ 3.8 MB/s | 1.6 MB 00:00 2024-04-23T01:42:38.157 INFO:teuthology.orchestra.run.smithi090.stdout:(3/36): ceph-mds-19.0.0-3192.ga1f34943.el7.x86_ 4.8 MB/s | 2.2 MB 00:00 2024-04-23T01:42:38.175 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-04-23T01:42:38.240 INFO:teuthology.orchestra.run.smithi090.stdout:(4/36): ceph-mgr-modules-core-19.0.0-3192.ga1f3 2.9 MB/s | 249 kB 00:00 2024-04-23T01:42:38.254 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-04-23T01:42:38.295 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-04-23T01:42:38.330 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-04-23T01:42:38.369 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-04-23T01:42:38.403 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-04-23T01:42:38.436 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-04-23T01:42:38.477 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-04-23T01:42:38.482 INFO:teuthology.orchestra.run.smithi090.stdout:(5/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.0 MB/s | 253 kB 00:00 2024-04-23T01:42:38.520 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-04-23T01:42:38.555 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-04-23T01:42:38.593 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-04-23T01:42:38.616 INFO:teuthology.orchestra.run.smithi090.stdout:(6/36): python3-cryptography-36.0.1-4.el9.x86_6 9.2 MB/s | 1.2 MB 00:00 2024-04-23T01:42:38.666 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-04-23T01:42:38.683 INFO:teuthology.orchestra.run.smithi090.stdout:(7/36): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-23T01:42:38.742 INFO:teuthology.orchestra.run.smithi090.stdout:(8/36): python3-pycparser-2.20-6.el9.noarch.rpm 2.3 MB/s | 135 kB 00:00 2024-04-23T01:42:38.803 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-04-23T01:42:38.808 INFO:teuthology.orchestra.run.smithi090.stdout:(9/36): python3-requests-2.25.1-8.el9.noarch.rp 1.8 MB/s | 125 kB 00:00 2024-04-23T01:42:38.875 INFO:teuthology.orchestra.run.smithi090.stdout:(10/36): python3-urllib3-1.26.5-5.el9.noarch.rp 3.1 MB/s | 215 kB 00:00 2024-04-23T01:42:38.883 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-04-23T01:42:38.945 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-04-23T01:42:38.984 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-04-23T01:42:38.985 INFO:teuthology.orchestra.run.smithi090.stdout:(11/36): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-04-23T01:42:39.033 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-04-23T01:42:39.051 INFO:teuthology.orchestra.run.smithi090.stdout:(12/36): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-04-23T01:42:39.088 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-04-23T01:42:39.102 INFO:teuthology.orchestra.run.smithi090.stdout:(13/36): python3-markupsafe-1.1.1-12.el9.x86_64 696 kB/s | 35 kB 00:00 2024-04-23T01:42:39.160 INFO:teuthology.orchestra.run.smithi090.stdout:(14/36): python3-pytz-2021.1-5.el9.noarch.rpm 879 kB/s | 51 kB 00:00 2024-04-23T01:42:39.219 INFO:teuthology.orchestra.run.smithi090.stdout:(15/36): python3-toml-0.10.2-6.el9.noarch.rpm 718 kB/s | 42 kB 00:00 2024-04-23T01:42:39.274 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-04-23T01:42:39.311 INFO:teuthology.orchestra.run.smithi090.stdout:(16/36): lua-devel-5.4.4-4.el9.x86_64.rpm 244 kB/s | 22 kB 00:00 2024-04-23T01:42:39.321 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-04-23T01:42:39.392 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-04-23T01:42:39.479 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-04-23T01:42:39.512 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-04-23T01:42:39.528 INFO:teuthology.orchestra.run.smithi090.stdout:(17/36): luarocks-3.9.2-1.el9.noarch.rpm 697 kB/s | 151 kB 00:00 2024-04-23T01:42:39.586 INFO:teuthology.orchestra.run.smithi090.stdout:(18/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 748 kB/s | 43 kB 00:00 2024-04-23T01:42:39.628 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-04-23T01:42:39.717 INFO:teuthology.orchestra.run.smithi090.stdout:(19/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-04-23T01:42:39.812 INFO:teuthology.orchestra.run.smithi090.stdout:(20/36): python3-cherrypy-18.6.1-2.el9.noarch.r 3.7 MB/s | 358 kB 00:00 2024-04-23T01:42:39.820 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3192.ga1f34943.el7. 31/36 2024-04-23T01:42:39.856 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-2:19.0.0-3192.ga1f34943.el7.x86_64 32/36 2024-04-23T01:42:39.879 INFO:teuthology.orchestra.run.smithi090.stdout:(21/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 160 kB/s | 11 kB 00:00 2024-04-23T01:42:39.881 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3192.ga1f34943.el7.x86_64 32/36 2024-04-23T01:42:39.881 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T01:42:39.881 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-23T01:42:39.881 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T01:42:39.881 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T01:42:39.881 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:42:39.939 INFO:teuthology.orchestra.run.smithi090.stdout:(22/36): python3-jaraco-classes-3.2.1-5.el9.noa 296 kB/s | 18 kB 00:00 2024-04-23T01:42:39.989 INFO:teuthology.orchestra.run.smithi090.stdout:(23/36): python3-jaraco-collections-3.0.0-8.el9 462 kB/s | 23 kB 00:00 2024-04-23T01:42:40.040 INFO:teuthology.orchestra.run.smithi090.stdout:(24/36): python3-jaraco-functools-3.5.0-2.el9.n 389 kB/s | 19 kB 00:00 2024-04-23T01:42:40.090 INFO:teuthology.orchestra.run.smithi090.stdout:(25/36): python3-jaraco-text-3.2.0-6.el9.noarch 393 kB/s | 20 kB 00:00 2024-04-23T01:42:40.149 INFO:teuthology.orchestra.run.smithi090.stdout:(26/36): python3-logutils-0.3.5-21.el9.noarch.r 793 kB/s | 46 kB 00:00 2024-04-23T01:42:40.208 INFO:teuthology.orchestra.run.smithi090.stdout:(27/36): python3-more-itertools-8.12.0-2.el9.no 1.3 MB/s | 79 kB 00:00 2024-04-23T01:42:40.275 INFO:teuthology.orchestra.run.smithi090.stdout:(28/36): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-04-23T01:42:40.341 INFO:teuthology.orchestra.run.smithi090.stdout:(29/36): python3-portend-3.1.0-2.el9.noarch.rpm 246 kB/s | 16 kB 00:00 2024-04-23T01:42:40.400 INFO:teuthology.orchestra.run.smithi090.stdout:(30/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.5 MB/s | 90 kB 00:00 2024-04-23T01:42:40.459 INFO:teuthology.orchestra.run.smithi090.stdout:(31/36): python3-tempora-5.0.0-2.el9.noarch.rpm 614 kB/s | 36 kB 00:00 2024-04-23T01:42:40.534 INFO:teuthology.orchestra.run.smithi090.stdout:(32/36): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-23T01:42:40.610 INFO:teuthology.orchestra.run.smithi090.stdout:(33/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 5.6 MB/s | 427 kB 00:00 2024-04-23T01:42:40.685 INFO:teuthology.orchestra.run.smithi090.stdout:(34/36): ceph-mon-19.0.0-3192.ga1f34943.el7.x86 1.9 MB/s | 5.0 MB 00:02 2024-04-23T01:42:40.710 INFO:teuthology.orchestra.run.smithi090.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 199 kB/s | 20 kB 00:00 2024-04-23T01:42:41.502 INFO:teuthology.orchestra.run.smithi090.stdout:(36/36): ceph-osd-19.0.0-3192.ga1f34943.el7.x86 21 MB/s | 71 MB 00:03 2024-04-23T01:42:41.506 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:42:41.506 INFO:teuthology.orchestra.run.smithi090.stdout:Total 19 MB/s | 85 MB 00:04 2024-04-23T01:42:41.649 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-23T01:42:41.689 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-23T01:42:41.690 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-23T01:42:42.302 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-23T01:42:42.302 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-23T01:42:43.858 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-23T01:42:43.922 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-04-23T01:42:43.954 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-04-23T01:42:43.991 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-04-23T01:42:44.061 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-04-23T01:42:44.108 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-04-23T01:42:44.149 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-04-23T01:42:44.241 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-04-23T01:42:44.310 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-04-23T01:42:44.352 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-04-23T01:42:44.396 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-04-23T01:42:44.434 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-04-23T01:42:44.468 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-04-23T01:42:44.502 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-04-23T01:42:44.542 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-04-23T01:42:44.586 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-04-23T01:42:44.613 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-04-23T01:42:44.651 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-04-23T01:42:44.725 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-04-23T01:42:44.870 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-04-23T01:42:44.952 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-04-23T01:42:45.004 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-04-23T01:42:45.052 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-04-23T01:42:45.109 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-04-23T01:42:45.174 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-04-23T01:42:45.359 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-04-23T01:42:45.419 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-04-23T01:42:45.494 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-04-23T01:42:45.584 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-04-23T01:42:45.624 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-04-23T01:42:45.739 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-04-23T01:42:45.934 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3192.ga1f34943.el7. 31/36 2024-04-23T01:42:45.961 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-2:19.0.0-3192.ga1f34943.el7.x86_64 32/36 2024-04-23T01:42:45.986 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3192.ga1f34943.el7.x86_64 32/36 2024-04-23T01:42:45.987 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T01:42:45.987 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-23T01:42:45.987 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T01:42:45.987 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T01:42:45.987 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:42:50.730 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-osd-2:19.0.0-3192.ga1f34943.el7.x86_64 33/36 2024-04-23T01:42:50.756 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-osd-2:19.0.0-3192.ga1f34943.el7.x86_64 33/36 2024-04-23T01:42:50.756 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T01:42:50.757 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-23T01:42:50.757 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T01:42:50.757 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T01:42:50.757 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:42:51.297 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mon-2:19.0.0-3192.ga1f34943.el7.x86_64 34/36 2024-04-23T01:42:51.320 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mon-2:19.0.0-3192.ga1f34943.el7.x86_64 34/36 2024-04-23T01:42:51.320 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T01:42:51.320 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-23T01:42:51.321 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T01:42:51.321 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T01:42:51.321 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:42:51.556 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mds-2:19.0.0-3192.ga1f34943.el7.x86_64 35/36 2024-04-23T01:42:51.579 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mds-2:19.0.0-3192.ga1f34943.el7.x86_64 35/36 2024-04-23T01:42:51.579 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T01:42:51.579 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-23T01:42:51.579 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T01:42:51.579 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T01:42:51.579 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:42:51.644 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-2:19.0.0-3192.ga1f34943.el7.x86_64 36/36 2024-04-23T01:42:56.904 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-osd-2:19.0.0-3192.ga1f34943.el7.x86_64 33/36 2024-04-23T01:42:56.931 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-osd-2:19.0.0-3192.ga1f34943.el7.x86_64 33/36 2024-04-23T01:42:56.931 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T01:42:56.931 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-23T01:42:56.931 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T01:42:56.931 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T01:42:56.931 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:42:57.505 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mon-2:19.0.0-3192.ga1f34943.el7.x86_64 34/36 2024-04-23T01:42:57.527 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mon-2:19.0.0-3192.ga1f34943.el7.x86_64 34/36 2024-04-23T01:42:57.527 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T01:42:57.527 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-23T01:42:57.528 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T01:42:57.528 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T01:42:57.528 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:42:57.764 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mds-2:19.0.0-3192.ga1f34943.el7.x86_64 35/36 2024-04-23T01:42:57.787 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mds-2:19.0.0-3192.ga1f34943.el7.x86_64 35/36 2024-04-23T01:42:57.787 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T01:42:57.787 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-23T01:42:57.787 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T01:42:57.787 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T01:42:57.787 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:42:57.843 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-2:19.0.0-3192.ga1f34943.el7.x86_64 36/36 2024-04-23T01:42:58.514 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-2:19.0.0-3192.ga1f34943.el7.x86_64 36/36 2024-04-23T01:42:58.515 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-2:19.0.0-3192.ga1f34943.el7.x86_64 1/36 2024-04-23T01:42:58.515 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mds-2:19.0.0-3192.ga1f34943.el7.x86_64 2/36 2024-04-23T01:42:58.515 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-2:19.0.0-3192.ga1f34943.el7.x86_64 3/36 2024-04-23T01:42:58.515 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mon-2:19.0.0-3192.ga1f34943.el7.x86_64 4/36 2024-04-23T01:42:58.515 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-osd-2:19.0.0-3192.ga1f34943.el7.x86_64 5/36 2024-04-23T01:42:58.515 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3192.ga1f34943.el7. 6/36 2024-04-23T01:42:58.515 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-04-23T01:42:58.515 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-04-23T01:42:58.515 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-04-23T01:42:58.515 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-04-23T01:42:58.515 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-04-23T01:42:58.515 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-04-23T01:42:58.515 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-04-23T01:42:58.516 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-04-23T01:42:58.516 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-04-23T01:42:58.516 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-04-23T01:42:58.516 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-04-23T01:42:58.516 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-04-23T01:42:58.516 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-04-23T01:42:58.516 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-04-23T01:42:58.516 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-04-23T01:42:58.516 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-04-23T01:42:58.516 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-04-23T01:42:58.516 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-04-23T01:42:58.516 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-04-23T01:42:58.516 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-04-23T01:42:58.516 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-04-23T01:42:58.517 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-04-23T01:42:58.517 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-04-23T01:42:58.517 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-04-23T01:42:58.517 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-04-23T01:42:58.517 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-04-23T01:42:58.517 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-04-23T01:42:58.517 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-04-23T01:42:58.517 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-04-23T01:42:59.343 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-04-23T01:42:59.343 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:42:59.344 INFO:teuthology.orchestra.run.smithi113.stdout:Installed: 2024-04-23T01:42:59.344 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:42:59.344 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mds-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:42:59.344 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:42:59.344 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-modules-core-2:19.0.0-3192.ga1f34943.el7.noarch 2024-04-23T01:42:59.344 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mon-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:42:59.344 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-osd-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:42:59.344 INFO:teuthology.orchestra.run.smithi113.stdout: lua-5.4.4-4.el9.x86_64 2024-04-23T01:42:59.344 INFO:teuthology.orchestra.run.smithi113.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-23T01:42:59.344 INFO:teuthology.orchestra.run.smithi113.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-23T01:42:59.344 INFO:teuthology.orchestra.run.smithi113.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-23T01:42:59.344 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-23T01:42:59.345 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-23T01:42:59.345 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-23T01:42:59.345 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-23T01:42:59.345 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-23T01:42:59.345 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-23T01:42:59.345 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-23T01:42:59.345 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-23T01:42:59.345 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-23T01:42:59.345 INFO:teuthology.orchestra.run.smithi113.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-23T01:42:59.345 INFO:teuthology.orchestra.run.smithi113.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-23T01:42:59.345 INFO:teuthology.orchestra.run.smithi113.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-23T01:42:59.345 INFO:teuthology.orchestra.run.smithi113.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-23T01:42:59.345 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-23T01:42:59.346 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ply-3.11-14.el9.noarch 2024-04-23T01:42:59.346 INFO:teuthology.orchestra.run.smithi113.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-23T01:42:59.346 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-23T01:42:59.346 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-23T01:42:59.346 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-23T01:42:59.346 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-23T01:42:59.346 INFO:teuthology.orchestra.run.smithi113.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-23T01:42:59.346 INFO:teuthology.orchestra.run.smithi113.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-23T01:42:59.346 INFO:teuthology.orchestra.run.smithi113.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-23T01:42:59.346 INFO:teuthology.orchestra.run.smithi113.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-23T01:42:59.346 INFO:teuthology.orchestra.run.smithi113.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-23T01:42:59.346 INFO:teuthology.orchestra.run.smithi113.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-23T01:42:59.346 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:42:59.346 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-23T01:42:59.685 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install ceph-base 2024-04-23T01:43:00.187 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:01:29 ago on Tue 23 Apr 2024 01:41:31 AM UTC. 2024-04-23T01:43:00.264 INFO:teuthology.orchestra.run.smithi113.stdout:Package ceph-base-2:19.0.0-3192.ga1f34943.el7.x86_64 is already installed. 2024-04-23T01:43:00.306 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-23T01:43:00.307 INFO:teuthology.orchestra.run.smithi113.stdout:Nothing to do. 2024-04-23T01:43:00.307 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-23T01:43:00.382 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install cephadm 2024-04-23T01:43:00.886 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:01:29 ago on Tue 23 Apr 2024 01:41:31 AM UTC. 2024-04-23T01:43:01.005 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-23T01:43:01.006 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:43:01.006 INFO:teuthology.orchestra.run.smithi113.stdout: Package Arch Version Repository Size 2024-04-23T01:43:01.006 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:43:01.006 INFO:teuthology.orchestra.run.smithi113.stdout:Installing: 2024-04-23T01:43:01.006 INFO:teuthology.orchestra.run.smithi113.stdout: cephadm noarch 2:19.0.0-3192.ga1f34943.el7 ceph-noarch 762 k 2024-04-23T01:43:01.006 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:43:01.006 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-04-23T01:43:01.006 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:43:01.006 INFO:teuthology.orchestra.run.smithi113.stdout:Install 1 Package 2024-04-23T01:43:01.006 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:43:01.007 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 762 k 2024-04-23T01:43:01.007 INFO:teuthology.orchestra.run.smithi113.stdout:Installed size: 769 k 2024-04-23T01:43:01.007 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2024-04-23T01:43:01.323 INFO:teuthology.orchestra.run.smithi113.stdout:cephadm-19.0.0-3192.ga1f34943.el7.noarch.rpm 2.4 MB/s | 762 kB 00:00 2024-04-23T01:43:01.323 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:43:01.323 INFO:teuthology.orchestra.run.smithi113.stdout:Total 2.3 MB/s | 762 kB 00:00 2024-04-23T01:43:01.324 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2024-04-23T01:43:01.329 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2024-04-23T01:43:01.329 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2024-04-23T01:43:01.334 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2024-04-23T01:43:01.335 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2024-04-23T01:43:01.408 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2024-04-23T01:43:01.873 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: cephadm-2:19.0.0-3192.ga1f34943.el7.noarch 1/1 2024-04-23T01:43:01.959 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephadm-2:19.0.0-3192.ga1f34943.el7.noarch 1/1 2024-04-23T01:43:02.318 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: cephadm-2:19.0.0-3192.ga1f34943.el7.noarch 1/1 2024-04-23T01:43:02.548 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephadm-2:19.0.0-3192.ga1f34943.el7.noarch 1/1 2024-04-23T01:43:02.549 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:43:02.549 INFO:teuthology.orchestra.run.smithi113.stdout:Installed: 2024-04-23T01:43:02.549 INFO:teuthology.orchestra.run.smithi113.stdout: cephadm-2:19.0.0-3192.ga1f34943.el7.noarch 2024-04-23T01:43:02.549 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:43:02.549 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-23T01:43:02.785 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install ceph-immutable-object-cache 2024-04-23T01:43:03.296 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:01:32 ago on Tue 23 Apr 2024 01:41:31 AM UTC. 2024-04-23T01:43:03.416 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-23T01:43:03.417 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:43:03.417 INFO:teuthology.orchestra.run.smithi113.stdout: Package Arch Version Repo Size 2024-04-23T01:43:03.417 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:43:03.417 INFO:teuthology.orchestra.run.smithi113.stdout:Installing: 2024-04-23T01:43:03.417 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 157 k 2024-04-23T01:43:03.417 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:43:03.418 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-04-23T01:43:03.418 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:43:03.418 INFO:teuthology.orchestra.run.smithi113.stdout:Install 1 Package 2024-04-23T01:43:03.418 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:43:03.418 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 157 k 2024-04-23T01:43:03.418 INFO:teuthology.orchestra.run.smithi113.stdout:Installed size: 640 k 2024-04-23T01:43:03.418 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2024-04-23T01:43:03.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-immutable-object-cache-19.0.0-3192.ga1f349 456 kB/s | 157 kB 00:00 2024-04-23T01:43:03.763 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:43:03.764 INFO:teuthology.orchestra.run.smithi113.stdout:Total 453 kB/s | 157 kB 00:00 2024-04-23T01:43:03.764 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2024-04-23T01:43:03.773 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2024-04-23T01:43:03.773 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2024-04-23T01:43:03.830 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2024-04-23T01:43:03.830 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2024-04-23T01:43:03.985 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2024-04-23T01:43:04.081 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3192.ga1f34943. 1/1 2024-04-23T01:43:04.105 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3192.ga1f34943. 1/1 2024-04-23T01:43:04.105 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T01:43:04.105 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-23T01:43:04.105 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:43:05.009 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3192.ga1f34943. 1/1 2024-04-23T01:43:05.009 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:43:05.009 INFO:teuthology.orchestra.run.smithi113.stdout:Installed: 2024-04-23T01:43:05.010 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-immutable-object-cache-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:43:05.010 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:43:05.010 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-23T01:43:05.230 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install ceph-mgr 2024-04-23T01:43:05.740 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:01:34 ago on Tue 23 Apr 2024 01:41:31 AM UTC. 2024-04-23T01:43:05.820 INFO:teuthology.orchestra.run.smithi113.stdout:Package ceph-mgr-2:19.0.0-3192.ga1f34943.el7.x86_64 is already installed. 2024-04-23T01:43:05.864 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-23T01:43:05.865 INFO:teuthology.orchestra.run.smithi113.stdout:Nothing to do. 2024-04-23T01:43:05.865 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-23T01:43:05.950 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install ceph-mgr-dashboard 2024-04-23T01:43:06.456 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:01:35 ago on Tue 23 Apr 2024 01:41:31 AM UTC. 2024-04-23T01:43:06.492 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-2:19.0.0-3192.ga1f34943.el7.x86_64 36/36 2024-04-23T01:43:06.492 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-2:19.0.0-3192.ga1f34943.el7.x86_64 1/36 2024-04-23T01:43:06.492 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mds-2:19.0.0-3192.ga1f34943.el7.x86_64 2/36 2024-04-23T01:43:06.493 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-2:19.0.0-3192.ga1f34943.el7.x86_64 3/36 2024-04-23T01:43:06.493 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mon-2:19.0.0-3192.ga1f34943.el7.x86_64 4/36 2024-04-23T01:43:06.493 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-osd-2:19.0.0-3192.ga1f34943.el7.x86_64 5/36 2024-04-23T01:43:06.493 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3192.ga1f34943.el7. 6/36 2024-04-23T01:43:06.493 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-04-23T01:43:06.493 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-04-23T01:43:06.493 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-04-23T01:43:06.493 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-04-23T01:43:06.493 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-04-23T01:43:06.494 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-04-23T01:43:06.494 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-04-23T01:43:06.494 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-04-23T01:43:06.494 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-04-23T01:43:06.494 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-04-23T01:43:06.494 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-04-23T01:43:06.494 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-04-23T01:43:06.494 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-04-23T01:43:06.494 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-04-23T01:43:06.494 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-04-23T01:43:06.494 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-04-23T01:43:06.494 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-04-23T01:43:06.494 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-04-23T01:43:06.495 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-04-23T01:43:06.495 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-04-23T01:43:06.495 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-04-23T01:43:06.496 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-04-23T01:43:06.496 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-04-23T01:43:06.496 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-04-23T01:43:06.497 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-04-23T01:43:06.497 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-04-23T01:43:06.497 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-04-23T01:43:06.497 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-04-23T01:43:06.497 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-04-23T01:43:06.575 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-23T01:43:06.576 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:43:06.576 INFO:teuthology.orchestra.run.smithi113.stdout: Package Arch Version Repository Size 2024-04-23T01:43:06.577 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:43:06.577 INFO:teuthology.orchestra.run.smithi113.stdout:Installing: 2024-04-23T01:43:06.577 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3192.ga1f34943.el7 ceph-noarch 3.5 M 2024-04-23T01:43:06.577 INFO:teuthology.orchestra.run.smithi113.stdout:Installing dependencies: 2024-04-23T01:43:06.577 INFO:teuthology.orchestra.run.smithi113.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-23T01:43:06.577 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3192.ga1f34943.el7 ceph-noarch 34 k 2024-04-23T01:43:06.577 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3192.ga1f34943.el7 ceph-noarch 16 k 2024-04-23T01:43:06.577 INFO:teuthology.orchestra.run.smithi113.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-23T01:43:06.577 INFO:teuthology.orchestra.run.smithi113.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-23T01:43:06.577 INFO:teuthology.orchestra.run.smithi113.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-23T01:43:06.577 INFO:teuthology.orchestra.run.smithi113.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-23T01:43:06.577 INFO:teuthology.orchestra.run.smithi113.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-23T01:43:06.577 INFO:teuthology.orchestra.run.smithi113.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-23T01:43:06.578 INFO:teuthology.orchestra.run.smithi113.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-23T01:43:06.578 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:43:06.578 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-04-23T01:43:06.578 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:43:06.578 INFO:teuthology.orchestra.run.smithi113.stdout:Install 11 Packages 2024-04-23T01:43:06.578 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:43:06.579 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 7.6 M 2024-04-23T01:43:06.579 INFO:teuthology.orchestra.run.smithi113.stdout:Installed size: 100 M 2024-04-23T01:43:06.579 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2024-04-23T01:43:07.325 INFO:teuthology.orchestra.run.smithi113.stdout:(1/11): ceph-grafana-dashboards-19.0.0-3192.ga1 173 kB/s | 34 kB 00:00 2024-04-23T01:43:07.350 INFO:teuthology.orchestra.run.smithi113.stdout:(2/11): ceph-prometheus-alerts-19.0.0-3192.ga1f 74 kB/s | 16 kB 00:00 2024-04-23T01:43:07.525 INFO:teuthology.orchestra.run.smithi113.stdout:(3/11): protobuf-compiler-3.14.0-13.el9.x86_64. 4.8 MB/s | 863 kB 00:00 2024-04-23T01:43:07.550 INFO:teuthology.orchestra.run.smithi113.stdout:(4/11): python3-protobuf-3.14.0-13.el9.noarch.r 1.2 MB/s | 269 kB 00:00 2024-04-23T01:43:07.590 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-04-23T01:43:07.590 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:07.590 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-23T01:43:07.591 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:43:07.591 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:43:07.591 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:43:07.591 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core-2:19.0.0-3192.ga1f34943.el7.noarch 2024-04-23T01:43:07.591 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:43:07.591 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-osd-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:43:07.591 INFO:teuthology.orchestra.run.smithi090.stdout: lua-5.4.4-4.el9.x86_64 2024-04-23T01:43:07.591 INFO:teuthology.orchestra.run.smithi090.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-23T01:43:07.591 INFO:teuthology.orchestra.run.smithi090.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-23T01:43:07.591 INFO:teuthology.orchestra.run.smithi090.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-23T01:43:07.591 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-23T01:43:07.591 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-23T01:43:07.592 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-23T01:43:07.592 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-23T01:43:07.592 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-23T01:43:07.592 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-23T01:43:07.592 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-23T01:43:07.592 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-23T01:43:07.592 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-23T01:43:07.592 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-23T01:43:07.592 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-23T01:43:07.592 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-23T01:43:07.592 INFO:teuthology.orchestra.run.smithi090.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-23T01:43:07.592 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-23T01:43:07.592 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ply-3.11-14.el9.noarch 2024-04-23T01:43:07.592 INFO:teuthology.orchestra.run.smithi090.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-23T01:43:07.593 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-23T01:43:07.593 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-23T01:43:07.593 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-23T01:43:07.593 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-23T01:43:07.593 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-23T01:43:07.593 INFO:teuthology.orchestra.run.smithi090.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-23T01:43:07.593 INFO:teuthology.orchestra.run.smithi090.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-23T01:43:07.593 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-23T01:43:07.593 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-23T01:43:07.593 INFO:teuthology.orchestra.run.smithi090.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-23T01:43:07.593 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:07.593 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-23T01:43:07.834 INFO:teuthology.orchestra.run.smithi113.stdout:(5/11): ceph-mgr-dashboard-19.0.0-3192.ga1f3494 5.0 MB/s | 3.5 MB 00:00 2024-04-23T01:43:07.901 INFO:teuthology.orchestra.run.smithi113.stdout:(6/11): grpc-data-1.46.7-10.el9.noarch.rpm 55 kB/s | 19 kB 00:00 2024-04-23T01:43:07.926 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-base 2024-04-23T01:43:08.193 INFO:teuthology.orchestra.run.smithi113.stdout:(7/11): python3-grpcio-tools-1.46.7-10.el9.x86_ 495 kB/s | 144 kB 00:00 2024-04-23T01:43:08.218 INFO:teuthology.orchestra.run.smithi113.stdout:(8/11): abseil-cpp-20211102.0-3.el9.x86_64.rpm 791 kB/s | 548 kB 00:00 2024-04-23T01:43:08.302 INFO:teuthology.orchestra.run.smithi113.stdout:(9/11): python3-repoze-lru-0.7-16.el9.noarch.rp 283 kB/s | 31 kB 00:00 2024-04-23T01:43:08.344 INFO:teuthology.orchestra.run.smithi113.stdout:(10/11): python3-routes-2.5.1-5.el9.noarch.rpm 1.5 MB/s | 188 kB 00:00 2024-04-23T01:43:08.426 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:41 ago on Tue 23 Apr 2024 01:41:27 AM UTC. 2024-04-23T01:43:08.503 INFO:teuthology.orchestra.run.smithi090.stdout:Package ceph-base-2:19.0.0-3192.ga1f34943.el7.x86_64 is already installed. 2024-04-23T01:43:08.545 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-23T01:43:08.546 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-23T01:43:08.546 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-23T01:43:08.632 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install cephadm 2024-04-23T01:43:08.719 INFO:teuthology.orchestra.run.smithi113.stdout:(11/11): python3-grpcio-1.46.7-10.el9.x86_64.rp 2.3 MB/s | 2.0 MB 00:00 2024-04-23T01:43:08.724 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:43:08.725 INFO:teuthology.orchestra.run.smithi113.stdout:Total 3.6 MB/s | 7.6 MB 00:02 2024-04-23T01:43:08.757 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2024-04-23T01:43:08.776 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2024-04-23T01:43:08.776 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2024-04-23T01:43:08.922 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2024-04-23T01:43:08.923 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2024-04-23T01:43:09.133 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:42 ago on Tue 23 Apr 2024 01:41:27 AM UTC. 2024-04-23T01:43:09.254 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-23T01:43:09.254 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:43:09.255 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-04-23T01:43:09.255 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:43:09.255 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-23T01:43:09.255 INFO:teuthology.orchestra.run.smithi090.stdout: cephadm noarch 2:19.0.0-3192.ga1f34943.el7 ceph-noarch 762 k 2024-04-23T01:43:09.255 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:09.255 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-23T01:43:09.255 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:43:09.255 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2024-04-23T01:43:09.255 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:09.255 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 762 k 2024-04-23T01:43:09.255 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 769 k 2024-04-23T01:43:09.256 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-23T01:43:09.400 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2024-04-23T01:43:09.492 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/11 2024-04-23T01:43:09.522 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/11 2024-04-23T01:43:09.539 INFO:teuthology.orchestra.run.smithi090.stdout:cephadm-19.0.0-3192.ga1f34943.el7.noarch.rpm 2.6 MB/s | 762 kB 00:00 2024-04-23T01:43:09.539 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:43:09.540 INFO:teuthology.orchestra.run.smithi090.stdout:Total 2.6 MB/s | 762 kB 00:00 2024-04-23T01:43:09.540 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-23T01:43:09.545 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-23T01:43:09.545 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-23T01:43:09.550 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-23T01:43:09.551 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-23T01:43:09.598 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 3/11 2024-04-23T01:43:09.625 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-23T01:43:09.749 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 4/11 2024-04-23T01:43:09.801 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 5/11 2024-04-23T01:43:09.918 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 6/11 2024-04-23T01:43:09.970 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 7/11 2024-04-23T01:43:10.007 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 8/11 2024-04-23T01:43:10.039 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3192.ga1f34943.el7 9/11 2024-04-23T01:43:10.075 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: cephadm-2:19.0.0-3192.ga1f34943.el7.noarch 1/1 2024-04-23T01:43:10.151 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephadm-2:19.0.0-3192.ga1f34943.el7.noarch 1/1 2024-04-23T01:43:10.555 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: cephadm-2:19.0.0-3192.ga1f34943.el7.noarch 1/1 2024-04-23T01:43:10.798 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephadm-2:19.0.0-3192.ga1f34943.el7.noarch 1/1 2024-04-23T01:43:10.798 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:10.798 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-23T01:43:10.799 INFO:teuthology.orchestra.run.smithi090.stdout: cephadm-2:19.0.0-3192.ga1f34943.el7.noarch 2024-04-23T01:43:10.799 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:10.799 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-23T01:43:11.070 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-immutable-object-cache 2024-04-23T01:43:11.079 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3192.ga1f34943.el 10/11 2024-04-23T01:43:11.114 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3192.ga1f34943.el7.noa 11/11 2024-04-23T01:43:11.591 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:44 ago on Tue 23 Apr 2024 01:41:27 AM UTC. 2024-04-23T01:43:11.712 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-23T01:43:11.713 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:43:11.713 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repo Size 2024-04-23T01:43:11.713 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:43:11.713 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-23T01:43:11.714 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 157 k 2024-04-23T01:43:11.714 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:11.714 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-23T01:43:11.714 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:43:11.714 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2024-04-23T01:43:11.714 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:11.714 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 157 k 2024-04-23T01:43:11.714 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 640 k 2024-04-23T01:43:11.714 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-23T01:43:11.934 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-immutable-object-cache-19.0.0-3192.ga1f349 717 kB/s | 157 kB 00:00 2024-04-23T01:43:11.934 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:43:11.935 INFO:teuthology.orchestra.run.smithi090.stdout:Total 710 kB/s | 157 kB 00:00 2024-04-23T01:43:11.935 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-23T01:43:11.944 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-23T01:43:11.945 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-23T01:43:12.000 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-23T01:43:12.001 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-23T01:43:12.153 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-23T01:43:12.243 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3192.ga1f34943. 1/1 2024-04-23T01:43:12.268 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3192.ga1f34943. 1/1 2024-04-23T01:43:12.268 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T01:43:12.268 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-23T01:43:12.268 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:12.415 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3192.ga1f34943.el7.noa 11/11 2024-04-23T01:43:12.415 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3192.ga1f34943.el 1/11 2024-04-23T01:43:12.415 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3192.ga1f34943.el7.noa 2/11 2024-04-23T01:43:12.415 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3192.ga1f34943.el7 3/11 2024-04-23T01:43:12.416 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 4/11 2024-04-23T01:43:12.416 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 5/11 2024-04-23T01:43:12.416 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 6/11 2024-04-23T01:43:12.416 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 7/11 2024-04-23T01:43:12.416 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 8/11 2024-04-23T01:43:12.416 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/11 2024-04-23T01:43:12.416 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 10/11 2024-04-23T01:43:12.700 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 11/11 2024-04-23T01:43:12.700 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:43:12.701 INFO:teuthology.orchestra.run.smithi113.stdout:Installed: 2024-04-23T01:43:12.701 INFO:teuthology.orchestra.run.smithi113.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-23T01:43:12.701 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-grafana-dashboards-2:19.0.0-3192.ga1f34943.el7.noarch 2024-04-23T01:43:12.701 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-dashboard-2:19.0.0-3192.ga1f34943.el7.noarch 2024-04-23T01:43:12.701 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-prometheus-alerts-2:19.0.0-3192.ga1f34943.el7.noarch 2024-04-23T01:43:12.701 INFO:teuthology.orchestra.run.smithi113.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-23T01:43:12.701 INFO:teuthology.orchestra.run.smithi113.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-23T01:43:12.701 INFO:teuthology.orchestra.run.smithi113.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-23T01:43:12.701 INFO:teuthology.orchestra.run.smithi113.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-23T01:43:12.701 INFO:teuthology.orchestra.run.smithi113.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-23T01:43:12.701 INFO:teuthology.orchestra.run.smithi113.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-23T01:43:12.701 INFO:teuthology.orchestra.run.smithi113.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-23T01:43:12.701 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:43:12.701 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-23T01:43:12.939 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-23T01:43:13.223 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3192.ga1f34943. 1/1 2024-04-23T01:43:13.224 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:13.224 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-23T01:43:13.224 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-immutable-object-cache-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:43:13.224 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:13.224 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-23T01:43:13.453 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:01:42 ago on Tue 23 Apr 2024 01:41:31 AM UTC. 2024-04-23T01:43:13.454 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-mgr 2024-04-23T01:43:13.575 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-23T01:43:13.576 INFO:teuthology.orchestra.run.smithi113.stdout:======================================================================================= 2024-04-23T01:43:13.576 INFO:teuthology.orchestra.run.smithi113.stdout: Package Arch Version Repository Size 2024-04-23T01:43:13.576 INFO:teuthology.orchestra.run.smithi113.stdout:======================================================================================= 2024-04-23T01:43:13.576 INFO:teuthology.orchestra.run.smithi113.stdout:Installing: 2024-04-23T01:43:13.576 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3192.ga1f34943.el7 ceph-noarch 7.4 M 2024-04-23T01:43:13.576 INFO:teuthology.orchestra.run.smithi113.stdout:Installing dependencies: 2024-04-23T01:43:13.576 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-23T01:43:13.576 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-23T01:43:13.576 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-23T01:43:13.577 INFO:teuthology.orchestra.run.smithi113.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-23T01:43:13.577 INFO:teuthology.orchestra.run.smithi113.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-23T01:43:13.577 INFO:teuthology.orchestra.run.smithi113.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-23T01:43:13.577 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-23T01:43:13.577 INFO:teuthology.orchestra.run.smithi113.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-23T01:43:13.577 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-23T01:43:13.577 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-23T01:43:13.577 INFO:teuthology.orchestra.run.smithi113.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-23T01:43:13.577 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:43:13.577 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-04-23T01:43:13.577 INFO:teuthology.orchestra.run.smithi113.stdout:======================================================================================= 2024-04-23T01:43:13.577 INFO:teuthology.orchestra.run.smithi113.stdout:Install 12 Packages 2024-04-23T01:43:13.577 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:43:13.578 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 38 M 2024-04-23T01:43:13.578 INFO:teuthology.orchestra.run.smithi113.stdout:Installed size: 209 M 2024-04-23T01:43:13.578 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2024-04-23T01:43:13.963 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:46 ago on Tue 23 Apr 2024 01:41:27 AM UTC. 2024-04-23T01:43:14.042 INFO:teuthology.orchestra.run.smithi090.stdout:Package ceph-mgr-2:19.0.0-3192.ga1f34943.el7.x86_64 is already installed. 2024-04-23T01:43:14.086 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-23T01:43:14.087 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-23T01:43:14.087 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-23T01:43:14.103 INFO:teuthology.orchestra.run.smithi113.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 680 kB/s | 195 kB 00:00 2024-04-23T01:43:14.142 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-mgr-dashboard 2024-04-23T01:43:14.304 INFO:teuthology.orchestra.run.smithi113.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 803 kB 00:00 2024-04-23T01:43:14.488 INFO:teuthology.orchestra.run.smithi113.stdout:(3/12): ceph-mgr-diskprediction-local-19.0.0-31 11 MB/s | 7.4 MB 00:00 2024-04-23T01:43:14.641 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:47 ago on Tue 23 Apr 2024 01:41:27 AM UTC. 2024-04-23T01:43:14.760 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-23T01:43:14.761 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:43:14.761 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-04-23T01:43:14.761 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:43:14.761 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-23T01:43:14.762 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3192.ga1f34943.el7 ceph-noarch 3.5 M 2024-04-23T01:43:14.762 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-04-23T01:43:14.762 INFO:teuthology.orchestra.run.smithi090.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-23T01:43:14.762 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3192.ga1f34943.el7 ceph-noarch 34 k 2024-04-23T01:43:14.762 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3192.ga1f34943.el7 ceph-noarch 16 k 2024-04-23T01:43:14.762 INFO:teuthology.orchestra.run.smithi090.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-23T01:43:14.762 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-23T01:43:14.762 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-23T01:43:14.762 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-23T01:43:14.762 INFO:teuthology.orchestra.run.smithi090.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-23T01:43:14.762 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-23T01:43:14.762 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-23T01:43:14.762 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:14.763 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-23T01:43:14.763 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:43:14.763 INFO:teuthology.orchestra.run.smithi090.stdout:Install 11 Packages 2024-04-23T01:43:14.763 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:14.763 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 7.6 M 2024-04-23T01:43:14.763 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 100 M 2024-04-23T01:43:14.763 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-23T01:43:14.806 INFO:teuthology.orchestra.run.smithi113.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 47 kB/s | 33 kB 00:00 2024-04-23T01:43:15.096 INFO:teuthology.orchestra.run.smithi113.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 30 kB/s | 18 kB 00:00 2024-04-23T01:43:15.208 INFO:teuthology.orchestra.run.smithi090.stdout:(1/11): ceph-grafana-dashboards-19.0.0-3192.ga1 159 kB/s | 34 kB 00:00 2024-04-23T01:43:15.349 INFO:teuthology.orchestra.run.smithi090.stdout:(2/11): python3-protobuf-3.14.0-13.el9.noarch.r 1.9 MB/s | 269 kB 00:00 2024-04-23T01:43:15.408 INFO:teuthology.orchestra.run.smithi090.stdout:(3/11): ceph-prometheus-alerts-19.0.0-3192.ga1f 40 kB/s | 16 kB 00:00 2024-04-23T01:43:15.530 INFO:teuthology.orchestra.run.smithi113.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 49 kB/s | 35 kB 00:00 2024-04-23T01:43:15.835 INFO:teuthology.orchestra.run.smithi090.stdout:(4/11): abseil-cpp-20211102.0-3.el9.x86_64.rpm 1.3 MB/s | 548 kB 00:00 2024-04-23T01:43:15.902 INFO:teuthology.orchestra.run.smithi090.stdout:(5/11): ceph-mgr-dashboard-19.0.0-3192.ga1f3494 3.9 MB/s | 3.5 MB 00:00 2024-04-23T01:43:15.927 INFO:teuthology.orchestra.run.smithi090.stdout:(6/11): grpc-data-1.46.7-10.el9.noarch.rpm 211 kB/s | 19 kB 00:00 2024-04-23T01:43:16.002 INFO:teuthology.orchestra.run.smithi090.stdout:(7/11): python3-grpcio-tools-1.46.7-10.el9.x86_ 1.9 MB/s | 144 kB 00:00 2024-04-23T01:43:16.069 INFO:teuthology.orchestra.run.smithi090.stdout:(8/11): python3-repoze-lru-0.7-16.el9.noarch.rp 462 kB/s | 31 kB 00:00 2024-04-23T01:43:16.145 INFO:teuthology.orchestra.run.smithi090.stdout:(9/11): python3-routes-2.5.1-5.el9.noarch.rpm 2.5 MB/s | 188 kB 00:00 2024-04-23T01:43:16.187 INFO:teuthology.orchestra.run.smithi090.stdout:(10/11): protobuf-compiler-3.14.0-13.el9.x86_64 1.0 MB/s | 863 kB 00:00 2024-04-23T01:43:16.370 INFO:teuthology.orchestra.run.smithi090.stdout:(11/11): python3-grpcio-1.46.7-10.el9.x86_64.rp 4.4 MB/s | 2.0 MB 00:00 2024-04-23T01:43:16.372 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:43:16.372 INFO:teuthology.orchestra.run.smithi090.stdout:Total 4.7 MB/s | 7.6 MB 00:01 2024-04-23T01:43:16.402 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-23T01:43:16.419 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-23T01:43:16.420 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-23T01:43:16.480 INFO:teuthology.orchestra.run.smithi113.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 257 kB/s | 244 kB 00:00 2024-04-23T01:43:16.566 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-23T01:43:16.566 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-23T01:43:17.039 INFO:teuthology.orchestra.run.smithi113.stdout:(8/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 2.4 MB/s | 4.7 MB 00:01 2024-04-23T01:43:17.044 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-23T01:43:17.135 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/11 2024-04-23T01:43:17.174 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/11 2024-04-23T01:43:17.249 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 3/11 2024-04-23T01:43:17.384 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 4/11 2024-04-23T01:43:17.438 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 5/11 2024-04-23T01:43:17.521 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 6/11 2024-04-23T01:43:17.573 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 7/11 2024-04-23T01:43:17.610 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 8/11 2024-04-23T01:43:17.651 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3192.ga1f34943.el7 9/11 2024-04-23T01:43:18.039 INFO:teuthology.orchestra.run.smithi113.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 251 kB/s | 250 kB 00:00 2024-04-23T01:43:18.702 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3192.ga1f34943.el 10/11 2024-04-23T01:43:18.743 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3192.ga1f34943.el7.noa 11/11 2024-04-23T01:43:19.122 INFO:teuthology.orchestra.run.smithi113.stdout:(10/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rp 635 kB/s | 3.0 MB 00:04 2024-04-23T01:43:20.157 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3192.ga1f34943.el7.noa 11/11 2024-04-23T01:43:20.157 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3192.ga1f34943.el 1/11 2024-04-23T01:43:20.157 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3192.ga1f34943.el7.noa 2/11 2024-04-23T01:43:20.157 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3192.ga1f34943.el7 3/11 2024-04-23T01:43:20.157 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 4/11 2024-04-23T01:43:20.157 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 5/11 2024-04-23T01:43:20.157 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 6/11 2024-04-23T01:43:20.158 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 7/11 2024-04-23T01:43:20.158 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 8/11 2024-04-23T01:43:20.158 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/11 2024-04-23T01:43:20.158 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 10/11 2024-04-23T01:43:20.448 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 11/11 2024-04-23T01:43:20.448 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:20.448 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-23T01:43:20.448 INFO:teuthology.orchestra.run.smithi090.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-23T01:43:20.448 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-grafana-dashboards-2:19.0.0-3192.ga1f34943.el7.noarch 2024-04-23T01:43:20.448 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard-2:19.0.0-3192.ga1f34943.el7.noarch 2024-04-23T01:43:20.448 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-prometheus-alerts-2:19.0.0-3192.ga1f34943.el7.noarch 2024-04-23T01:43:20.449 INFO:teuthology.orchestra.run.smithi090.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-23T01:43:20.449 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-23T01:43:20.449 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-23T01:43:20.449 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-23T01:43:20.449 INFO:teuthology.orchestra.run.smithi090.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-23T01:43:20.449 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-23T01:43:20.449 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-23T01:43:20.449 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:20.449 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-23T01:43:20.686 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-23T01:43:21.193 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:54 ago on Tue 23 Apr 2024 01:41:27 AM UTC. 2024-04-23T01:43:21.314 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-23T01:43:21.315 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================= 2024-04-23T01:43:21.315 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-04-23T01:43:21.315 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================= 2024-04-23T01:43:21.315 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-23T01:43:21.315 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3192.ga1f34943.el7 ceph-noarch 7.4 M 2024-04-23T01:43:21.315 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-04-23T01:43:21.315 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-23T01:43:21.316 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-23T01:43:21.316 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-23T01:43:21.316 INFO:teuthology.orchestra.run.smithi090.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-23T01:43:21.316 INFO:teuthology.orchestra.run.smithi090.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-23T01:43:21.316 INFO:teuthology.orchestra.run.smithi090.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-23T01:43:21.316 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-23T01:43:21.316 INFO:teuthology.orchestra.run.smithi090.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-23T01:43:21.316 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-23T01:43:21.316 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-23T01:43:21.316 INFO:teuthology.orchestra.run.smithi090.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-23T01:43:21.316 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:21.316 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-23T01:43:21.316 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================= 2024-04-23T01:43:21.316 INFO:teuthology.orchestra.run.smithi090.stdout:Install 12 Packages 2024-04-23T01:43:21.317 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:21.317 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 38 M 2024-04-23T01:43:21.317 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 209 M 2024-04-23T01:43:21.317 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-23T01:43:21.841 INFO:teuthology.orchestra.run.smithi090.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-04-23T01:43:21.917 INFO:teuthology.orchestra.run.smithi090.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 803 kB 00:00 2024-04-23T01:43:21.942 INFO:teuthology.orchestra.run.smithi090.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 326 kB/s | 33 kB 00:00 2024-04-23T01:43:21.984 INFO:teuthology.orchestra.run.smithi090.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 436 kB/s | 18 kB 00:00 2024-04-23T01:43:22.034 INFO:teuthology.orchestra.run.smithi090.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 704 kB/s | 35 kB 00:00 2024-04-23T01:43:22.170 INFO:teuthology.orchestra.run.smithi090.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 12 MB/s | 3.0 MB 00:00 2024-04-23T01:43:22.210 INFO:teuthology.orchestra.run.smithi090.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-04-23T01:43:22.285 INFO:teuthology.orchestra.run.smithi090.stdout:(8/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 19 MB/s | 4.7 MB 00:00 2024-04-23T01:43:22.336 INFO:teuthology.orchestra.run.smithi090.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 4.9 MB/s | 250 kB 00:00 2024-04-23T01:43:22.436 INFO:teuthology.orchestra.run.smithi090.stdout:(10/12): ceph-mgr-diskprediction-local-19.0.0-3 9.4 MB/s | 7.4 MB 00:00 2024-04-23T01:43:22.511 INFO:teuthology.orchestra.run.smithi090.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-04-23T01:43:22.770 INFO:teuthology.orchestra.run.smithi090.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-04-23T01:43:22.771 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:43:22.771 INFO:teuthology.orchestra.run.smithi090.stdout:Total 26 MB/s | 38 MB 00:01 2024-04-23T01:43:23.115 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-23T01:43:23.140 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-23T01:43:23.140 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-23T01:43:23.543 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-23T01:43:23.544 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-23T01:43:23.663 INFO:teuthology.orchestra.run.smithi113.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 727 kB/s | 5.1 MB 00:07 2024-04-23T01:43:24.376 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-23T01:43:24.442 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-23T01:43:24.486 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-23T01:43:24.584 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-23T01:43:24.629 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-23T01:43:24.806 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-23T01:43:24.836 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-23T01:43:24.921 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-23T01:43:25.556 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-23T01:43:25.701 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-23T01:43:26.910 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-23T01:43:27.949 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-23T01:43:27.985 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3192.ga1f34 12/12 2024-04-23T01:43:31.191 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3192.ga1f34 12/12 2024-04-23T01:43:31.192 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3192.ga1f34 1/12 2024-04-23T01:43:31.192 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-23T01:43:31.192 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-23T01:43:31.192 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-23T01:43:31.192 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-23T01:43:31.192 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-23T01:43:31.192 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-23T01:43:31.192 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-23T01:43:31.192 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-23T01:43:31.192 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-23T01:43:31.192 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-23T01:43:31.585 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-23T01:43:31.585 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:31.585 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-23T01:43:31.585 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3192.ga1f34943.el7.noarch 2024-04-23T01:43:31.586 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-23T01:43:31.586 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-23T01:43:31.586 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-23T01:43:31.586 INFO:teuthology.orchestra.run.smithi090.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-23T01:43:31.586 INFO:teuthology.orchestra.run.smithi090.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-23T01:43:31.586 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-23T01:43:31.586 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-23T01:43:31.586 INFO:teuthology.orchestra.run.smithi090.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-23T01:43:31.586 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-23T01:43:31.586 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-23T01:43:31.586 INFO:teuthology.orchestra.run.smithi090.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-23T01:43:31.586 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:31.586 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-23T01:43:32.116 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-mgr-rook 2024-04-23T01:43:32.641 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:02:05 ago on Tue 23 Apr 2024 01:41:27 AM UTC. 2024-04-23T01:43:32.762 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-23T01:43:32.763 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:43:32.763 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-04-23T01:43:32.763 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:43:32.763 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-23T01:43:32.763 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-rook noarch 2:19.0.0-3192.ga1f34943.el7 ceph-noarch 49 k 2024-04-23T01:43:32.763 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-04-23T01:43:32.764 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-23T01:43:32.764 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-23T01:43:32.764 INFO:teuthology.orchestra.run.smithi090.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-23T01:43:32.764 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-23T01:43:32.764 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-23T01:43:32.764 INFO:teuthology.orchestra.run.smithi090.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-23T01:43:32.764 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-23T01:43:32.764 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-23T01:43:32.764 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-23T01:43:32.764 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-23T01:43:32.764 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-23T01:43:32.764 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-23T01:43:32.764 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:32.764 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-23T01:43:32.765 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:43:32.765 INFO:teuthology.orchestra.run.smithi090.stdout:Install 13 Packages 2024-04-23T01:43:32.765 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:32.765 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 2.2 M 2024-04-23T01:43:32.765 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 26 M 2024-04-23T01:43:32.765 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-23T01:43:33.116 INFO:teuthology.orchestra.run.smithi090.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 275 kB/s | 26 kB 00:00 2024-04-23T01:43:33.142 INFO:teuthology.orchestra.run.smithi090.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 162 kB/s | 19 kB 00:00 2024-04-23T01:43:33.283 INFO:teuthology.orchestra.run.smithi090.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.5 MB/s | 222 kB 00:00 2024-04-23T01:43:33.309 INFO:teuthology.orchestra.run.smithi090.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 955 kB/s | 159 kB 00:00 2024-04-23T01:43:33.334 INFO:teuthology.orchestra.run.smithi090.stdout:(5/13): ceph-mgr-rook-19.0.0-3192.ga1f34943.el7 158 kB/s | 49 kB 00:00 2024-04-23T01:43:33.368 INFO:teuthology.orchestra.run.smithi090.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 915 kB/s | 54 kB 00:00 2024-04-23T01:43:33.401 INFO:teuthology.orchestra.run.smithi090.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.3 MB/s | 279 kB 00:00 2024-04-23T01:43:33.602 INFO:teuthology.orchestra.run.smithi090.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 120 kB/s | 32 kB 00:00 2024-04-23T01:43:33.637 INFO:teuthology.orchestra.run.smithi090.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 56 kB/s | 15 kB 00:00 2024-04-23T01:43:33.837 INFO:teuthology.orchestra.run.smithi090.stdout:(10/13): python3-google-auth-2.28.2-1.el9.noarc 466 kB/s | 203 kB 00:00 2024-04-23T01:43:33.987 INFO:teuthology.orchestra.run.smithi090.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 169 kB/s | 59 kB 00:00 2024-04-23T01:43:34.188 INFO:teuthology.orchestra.run.smithi090.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 256 kB/s | 90 kB 00:00 2024-04-23T01:43:34.304 INFO:teuthology.orchestra.run.smithi090.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.5 MB/s | 1.0 MB 00:00 2024-04-23T01:43:34.306 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:43:34.306 INFO:teuthology.orchestra.run.smithi090.stdout:Total 1.4 MB/s | 2.2 MB 00:01 2024-04-23T01:43:34.379 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-23T01:43:34.396 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-23T01:43:34.396 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-23T01:43:34.490 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-23T01:43:34.490 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-23T01:43:34.999 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-23T01:43:35.179 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-23T01:43:35.240 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-23T01:43:35.303 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-23T01:43:35.341 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-23T01:43:35.382 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-23T01:43:35.476 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-23T01:43:35.586 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-04-23T01:43:35.644 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-23T01:43:36.354 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-23T01:43:36.417 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-23T01:43:36.450 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-23T01:43:36.499 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-23T01:43:36.529 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-rook-2:19.0.0-3192.ga1f34943.el7.noarch 13/13 2024-04-23T01:43:37.451 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3192.ga1f34943.el7.noarch 13/13 2024-04-23T01:43:37.451 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3192.ga1f34943.el7.noarch 1/13 2024-04-23T01:43:37.451 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-23T01:43:37.451 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-23T01:43:37.451 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-23T01:43:37.451 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-23T01:43:37.451 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-23T01:43:37.451 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-23T01:43:37.452 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-23T01:43:37.452 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-23T01:43:37.452 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-04-23T01:43:37.452 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-23T01:43:37.452 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-23T01:43:37.775 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-23T01:43:37.775 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:37.775 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-23T01:43:37.776 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-rook-2:19.0.0-3192.ga1f34943.el7.noarch 2024-04-23T01:43:37.776 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-23T01:43:37.776 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-23T01:43:37.776 INFO:teuthology.orchestra.run.smithi090.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-23T01:43:37.776 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-23T01:43:37.776 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-23T01:43:37.776 INFO:teuthology.orchestra.run.smithi090.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-23T01:43:37.776 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-23T01:43:37.776 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-23T01:43:37.776 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-23T01:43:37.776 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-23T01:43:37.776 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-23T01:43:37.776 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-23T01:43:37.776 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:37.776 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-23T01:43:38.016 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-mgr-cephadm 2024-04-23T01:43:38.524 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:02:11 ago on Tue 23 Apr 2024 01:41:27 AM UTC. 2024-04-23T01:43:38.644 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-23T01:43:38.645 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:43:38.645 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-04-23T01:43:38.645 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:43:38.645 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-23T01:43:38.645 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3192.ga1f34943.el7 ceph-noarch 142 k 2024-04-23T01:43:38.645 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-04-23T01:43:38.646 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-23T01:43:38.646 INFO:teuthology.orchestra.run.smithi090.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-23T01:43:38.646 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-23T01:43:38.646 INFO:teuthology.orchestra.run.smithi090.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-23T01:43:38.646 INFO:teuthology.orchestra.run.smithi090.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-23T01:43:38.646 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:38.646 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-23T01:43:38.646 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:43:38.646 INFO:teuthology.orchestra.run.smithi090.stdout:Install 6 Packages 2024-04-23T01:43:38.646 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:38.647 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 7.0 M 2024-04-23T01:43:38.647 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 33 M 2024-04-23T01:43:38.647 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-23T01:43:39.379 INFO:teuthology.orchestra.run.smithi090.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 824 kB/s | 248 kB 00:00 2024-04-23T01:43:39.460 INFO:teuthology.orchestra.run.smithi113.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 777 kB/s | 16 MB 00:21 2024-04-23T01:43:39.462 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:43:39.462 INFO:teuthology.orchestra.run.smithi113.stdout:Total 1.5 MB/s | 38 MB 00:25 2024-04-23T01:43:39.471 INFO:teuthology.orchestra.run.smithi090.stdout:(2/6): ceph-mgr-cephadm-19.0.0-3192.ga1f34943.e 362 kB/s | 142 kB 00:00 2024-04-23T01:43:39.647 INFO:teuthology.orchestra.run.smithi090.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-04-23T01:43:39.672 INFO:teuthology.orchestra.run.smithi090.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 288 kB/s | 58 kB 00:00 2024-04-23T01:43:39.805 INFO:teuthology.orchestra.run.smithi090.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.3 MB/s | 548 kB 00:00 2024-04-23T01:43:39.812 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2024-04-23T01:43:39.830 INFO:teuthology.orchestra.run.smithi090.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 278 kB/s | 51 kB 00:00 2024-04-23T01:43:39.833 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:43:39.833 INFO:teuthology.orchestra.run.smithi090.stdout:Total 5.9 MB/s | 7.0 MB 00:01 2024-04-23T01:43:39.839 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2024-04-23T01:43:39.839 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2024-04-23T01:43:39.902 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-23T01:43:39.913 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-23T01:43:39.913 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-23T01:43:40.008 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-23T01:43:40.008 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-23T01:43:40.246 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2024-04-23T01:43:40.246 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2024-04-23T01:43:40.305 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-23T01:43:40.489 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-23T01:43:40.541 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-23T01:43:41.052 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2024-04-23T01:43:41.065 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-23T01:43:41.123 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-23T01:43:41.158 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-23T01:43:41.195 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-23T01:43:41.258 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-23T01:43:41.266 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-23T01:43:41.305 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3192.ga1f34943.el7.noarch 6/6 2024-04-23T01:43:41.310 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-23T01:43:41.489 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-23T01:43:41.525 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-23T01:43:41.603 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-23T01:43:42.242 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-23T01:43:42.364 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-23T01:43:42.400 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3192.ga1f34943.el7.noarch 6/6 2024-04-23T01:43:42.400 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3192.ga1f34943.el7.noarch 1/6 2024-04-23T01:43:42.400 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-23T01:43:42.400 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-23T01:43:42.400 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-23T01:43:42.400 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-23T01:43:42.648 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-23T01:43:42.648 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:42.648 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-23T01:43:42.648 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-cephadm-2:19.0.0-3192.ga1f34943.el7.noarch 2024-04-23T01:43:42.648 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-23T01:43:42.648 INFO:teuthology.orchestra.run.smithi090.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-23T01:43:42.649 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-23T01:43:42.649 INFO:teuthology.orchestra.run.smithi090.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-23T01:43:42.649 INFO:teuthology.orchestra.run.smithi090.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-23T01:43:42.649 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:42.649 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-23T01:43:42.899 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-fuse 2024-04-23T01:43:43.411 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:02:16 ago on Tue 23 Apr 2024 01:41:27 AM UTC. 2024-04-23T01:43:43.531 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-23T01:43:43.532 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:43:43.532 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-04-23T01:43:43.532 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:43:43.532 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-23T01:43:43.532 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-fuse x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 871 k 2024-04-23T01:43:43.533 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-04-23T01:43:43.533 INFO:teuthology.orchestra.run.smithi090.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-23T01:43:43.533 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:43.533 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-23T01:43:43.533 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:43:43.533 INFO:teuthology.orchestra.run.smithi090.stdout:Install 2 Packages 2024-04-23T01:43:43.533 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:43.533 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 951 k 2024-04-23T01:43:43.533 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 4.0 M 2024-04-23T01:43:43.533 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-23T01:43:43.599 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-23T01:43:43.995 INFO:teuthology.orchestra.run.smithi090.stdout:(1/2): ceph-fuse-19.0.0-3192.ga1f34943.el7.x86_ 2.7 MB/s | 871 kB 00:00 2024-04-23T01:43:44.104 INFO:teuthology.orchestra.run.smithi090.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 191 kB/s | 80 kB 00:00 2024-04-23T01:43:44.104 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:43:44.105 INFO:teuthology.orchestra.run.smithi090.stdout:Total 1.6 MB/s | 951 kB 00:00 2024-04-23T01:43:44.117 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-23T01:43:44.142 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-23T01:43:44.142 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-23T01:43:44.220 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-23T01:43:44.220 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-23T01:43:44.401 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-23T01:43:44.547 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-23T01:43:44.627 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-fuse-2:19.0.0-3192.ga1f34943.el7.x86_64 2/2 2024-04-23T01:43:44.645 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-23T01:43:44.677 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3192.ga1f34 12/12 2024-04-23T01:43:45.329 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-fuse-2:19.0.0-3192.ga1f34943.el7.x86_64 2/2 2024-04-23T01:43:45.329 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-fuse-2:19.0.0-3192.ga1f34943.el7.x86_64 1/2 2024-04-23T01:43:45.600 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-23T01:43:45.600 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:45.600 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-23T01:43:45.600 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-fuse-2:19.0.0-3192.ga1f34943.el7.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-23T01:43:45.600 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:45.600 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-23T01:43:45.894 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-volume 2024-04-23T01:43:46.415 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:02:19 ago on Tue 23 Apr 2024 01:41:27 AM UTC. 2024-04-23T01:43:46.537 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-23T01:43:46.538 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:43:46.538 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-04-23T01:43:46.538 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:43:46.538 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-23T01:43:46.538 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-volume noarch 2:19.0.0-3192.ga1f34943.el7 ceph-noarch 273 k 2024-04-23T01:43:46.538 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-04-23T01:43:46.538 INFO:teuthology.orchestra.run.smithi090.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-23T01:43:46.538 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:46.538 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-23T01:43:46.538 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:43:46.539 INFO:teuthology.orchestra.run.smithi090.stdout:Install 2 Packages 2024-04-23T01:43:46.539 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:46.539 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 350 k 2024-04-23T01:43:46.539 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 1.5 M 2024-04-23T01:43:46.539 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-23T01:43:46.780 INFO:teuthology.orchestra.run.smithi090.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 772 kB/s | 77 kB 00:00 2024-04-23T01:43:46.961 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3192.ga1f34 12/12 2024-04-23T01:43:46.961 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3192.ga1f34 1/12 2024-04-23T01:43:46.961 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-23T01:43:46.961 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-23T01:43:46.962 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-23T01:43:46.962 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-23T01:43:46.962 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-23T01:43:46.962 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-23T01:43:46.962 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-23T01:43:46.962 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-23T01:43:46.962 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-23T01:43:46.962 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-23T01:43:47.039 INFO:teuthology.orchestra.run.smithi090.stdout:(2/2): ceph-volume-19.0.0-3192.ga1f34943.el7.no 761 kB/s | 273 kB 00:00 2024-04-23T01:43:47.040 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:43:47.040 INFO:teuthology.orchestra.run.smithi090.stdout:Total 699 kB/s | 350 kB 00:00 2024-04-23T01:43:47.051 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-23T01:43:47.061 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-23T01:43:47.061 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-23T01:43:47.087 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-23T01:43:47.087 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-23T01:43:47.229 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-23T01:43:47.375 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-23T01:43:47.375 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:43:47.375 INFO:teuthology.orchestra.run.smithi113.stdout:Installed: 2024-04-23T01:43:47.375 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3192.ga1f34943.el7.noarch 2024-04-23T01:43:47.375 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-23T01:43:47.375 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-23T01:43:47.375 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-23T01:43:47.375 INFO:teuthology.orchestra.run.smithi113.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-23T01:43:47.375 INFO:teuthology.orchestra.run.smithi113.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-23T01:43:47.375 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-23T01:43:47.375 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-23T01:43:47.376 INFO:teuthology.orchestra.run.smithi113.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-23T01:43:47.376 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-23T01:43:47.376 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-23T01:43:47.376 INFO:teuthology.orchestra.run.smithi113.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-23T01:43:47.376 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:43:47.376 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-23T01:43:47.418 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-04-23T01:43:47.450 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-volume-2:19.0.0-3192.ga1f34943.el7.noarch 2/2 2024-04-23T01:43:47.466 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-volume-2:19.0.0-3192.ga1f34943.el7.noarch 2/2 2024-04-23T01:43:47.466 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T01:43:47.466 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-23T01:43:47.466 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:47.789 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install ceph-mgr-rook 2024-04-23T01:43:48.220 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-volume-2:19.0.0-3192.ga1f34943.el7.noarch 1/2 2024-04-23T01:43:48.313 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:02:17 ago on Tue 23 Apr 2024 01:41:31 AM UTC. 2024-04-23T01:43:48.436 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-23T01:43:48.437 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:43:48.437 INFO:teuthology.orchestra.run.smithi113.stdout: Package Arch Version Repository Size 2024-04-23T01:43:48.437 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:43:48.437 INFO:teuthology.orchestra.run.smithi113.stdout:Installing: 2024-04-23T01:43:48.438 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-rook noarch 2:19.0.0-3192.ga1f34943.el7 ceph-noarch 49 k 2024-04-23T01:43:48.438 INFO:teuthology.orchestra.run.smithi113.stdout:Installing dependencies: 2024-04-23T01:43:48.438 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-23T01:43:48.438 INFO:teuthology.orchestra.run.smithi113.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-23T01:43:48.438 INFO:teuthology.orchestra.run.smithi113.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-23T01:43:48.438 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-23T01:43:48.438 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-23T01:43:48.438 INFO:teuthology.orchestra.run.smithi113.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-23T01:43:48.438 INFO:teuthology.orchestra.run.smithi113.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-23T01:43:48.438 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-23T01:43:48.438 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-23T01:43:48.438 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-23T01:43:48.438 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-23T01:43:48.439 INFO:teuthology.orchestra.run.smithi113.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-23T01:43:48.439 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:43:48.439 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-04-23T01:43:48.439 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:43:48.439 INFO:teuthology.orchestra.run.smithi113.stdout:Install 13 Packages 2024-04-23T01:43:48.439 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:43:48.439 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 2.2 M 2024-04-23T01:43:48.439 INFO:teuthology.orchestra.run.smithi113.stdout:Installed size: 26 M 2024-04-23T01:43:48.440 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2024-04-23T01:43:48.534 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-04-23T01:43:48.534 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:48.534 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-23T01:43:48.534 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-volume-2:19.0.0-3192.ga1f34943.el7.noarch 2024-04-23T01:43:48.534 INFO:teuthology.orchestra.run.smithi090.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-23T01:43:48.534 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:48.534 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-23T01:43:48.781 INFO:teuthology.orchestra.run.smithi113.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 303 kB/s | 26 kB 00:00 2024-04-23T01:43:48.806 INFO:teuthology.orchestra.run.smithi113.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 175 kB/s | 19 kB 00:00 2024-04-23T01:43:48.851 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install librados-devel 2024-04-23T01:43:48.948 INFO:teuthology.orchestra.run.smithi113.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-04-23T01:43:48.973 INFO:teuthology.orchestra.run.smithi113.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.3 MB/s | 222 kB 00:00 2024-04-23T01:43:48.998 INFO:teuthology.orchestra.run.smithi113.stdout:(5/13): ceph-mgr-rook-19.0.0-3192.ga1f34943.el7 163 kB/s | 49 kB 00:00 2024-04-23T01:43:49.024 INFO:teuthology.orchestra.run.smithi113.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-04-23T01:43:49.124 INFO:teuthology.orchestra.run.smithi113.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.5 MB/s | 279 kB 00:00 2024-04-23T01:43:49.183 INFO:teuthology.orchestra.run.smithi113.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 94 kB/s | 15 kB 00:00 2024-04-23T01:43:49.249 INFO:teuthology.orchestra.run.smithi113.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 128 kB/s | 32 kB 00:00 2024-04-23T01:43:49.367 INFO:teuthology.orchestra.run.smithi113.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 506 kB/s | 59 kB 00:00 2024-04-23T01:43:49.374 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:02:22 ago on Tue 23 Apr 2024 01:41:27 AM UTC. 2024-04-23T01:43:49.425 INFO:teuthology.orchestra.run.smithi113.stdout:(11/13): python3-google-auth-2.28.2-1.el9.noarc 673 kB/s | 203 kB 00:00 2024-04-23T01:43:49.451 INFO:teuthology.orchestra.run.smithi113.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.0 MB/s | 90 kB 00:00 2024-04-23T01:43:49.498 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-23T01:43:49.498 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:43:49.499 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repo Size 2024-04-23T01:43:49.499 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:43:49.499 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-23T01:43:49.499 INFO:teuthology.orchestra.run.smithi090.stdout: librados-devel x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 125 k 2024-04-23T01:43:49.499 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:49.499 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-23T01:43:49.499 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:43:49.499 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2024-04-23T01:43:49.499 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:49.500 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 125 k 2024-04-23T01:43:49.500 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 445 k 2024-04-23T01:43:49.500 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-23T01:43:49.534 INFO:teuthology.orchestra.run.smithi113.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 2.9 MB/s | 1.0 MB 00:00 2024-04-23T01:43:49.534 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:43:49.535 INFO:teuthology.orchestra.run.smithi113.stdout:Total 2.0 MB/s | 2.2 MB 00:01 2024-04-23T01:43:49.597 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2024-04-23T01:43:49.612 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2024-04-23T01:43:49.612 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2024-04-23T01:43:49.697 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2024-04-23T01:43:49.701 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2024-04-23T01:43:49.725 INFO:teuthology.orchestra.run.smithi090.stdout:librados-devel-19.0.0-3192.ga1f34943.el7.x86_64 557 kB/s | 125 kB 00:00 2024-04-23T01:43:49.725 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:43:49.725 INFO:teuthology.orchestra.run.smithi090.stdout:Total 552 kB/s | 125 kB 00:00 2024-04-23T01:43:49.726 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-23T01:43:49.731 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-23T01:43:49.731 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-23T01:43:49.794 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-23T01:43:49.794 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-23T01:43:49.945 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-23T01:43:50.045 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librados-devel-2:19.0.0-3192.ga1f34943.el7.x86_64 1/1 2024-04-23T01:43:50.196 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2024-04-23T01:43:50.383 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-23T01:43:50.444 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-23T01:43:50.499 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-23T01:43:50.526 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados-devel-2:19.0.0-3192.ga1f34943.el7.x86_64 1/1 2024-04-23T01:43:50.537 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-23T01:43:50.578 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-23T01:43:50.673 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-23T01:43:50.761 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados-devel-2:19.0.0-3192.ga1f34943.el7.x86_64 1/1 2024-04-23T01:43:50.761 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:50.761 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-23T01:43:50.762 INFO:teuthology.orchestra.run.smithi090.stdout: librados-devel-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:43:50.762 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:50.762 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-23T01:43:50.774 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-04-23T01:43:50.839 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-23T01:43:51.037 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install libcephfs2 2024-04-23T01:43:51.555 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:02:24 ago on Tue 23 Apr 2024 01:41:27 AM UTC. 2024-04-23T01:43:51.565 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-23T01:43:51.611 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-23T01:43:51.634 INFO:teuthology.orchestra.run.smithi090.stdout:Package libcephfs2-2:19.0.0-3192.ga1f34943.el7.x86_64 is already installed. 2024-04-23T01:43:51.645 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-23T01:43:51.676 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-23T01:43:51.677 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-23T01:43:51.678 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-23T01:43:51.701 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-23T01:43:51.723 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-rook-2:19.0.0-3192.ga1f34943.el7.noarch 13/13 2024-04-23T01:43:51.740 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install libcephfs-devel 2024-04-23T01:43:52.255 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:02:25 ago on Tue 23 Apr 2024 01:41:27 AM UTC. 2024-04-23T01:43:52.375 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-23T01:43:52.376 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:43:52.376 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repo Size 2024-04-23T01:43:52.376 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:43:52.376 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-23T01:43:52.376 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-devel x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 32 k 2024-04-23T01:43:52.376 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:52.376 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-23T01:43:52.376 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:43:52.376 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2024-04-23T01:43:52.376 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:52.376 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 32 k 2024-04-23T01:43:52.377 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 142 k 2024-04-23T01:43:52.377 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-23T01:43:52.454 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3192.ga1f34943.el7.noarch 13/13 2024-04-23T01:43:52.454 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3192.ga1f34943.el7.noarch 1/13 2024-04-23T01:43:52.455 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-23T01:43:52.455 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-23T01:43:52.455 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-23T01:43:52.455 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-23T01:43:52.455 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-23T01:43:52.455 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-23T01:43:52.455 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-23T01:43:52.455 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-23T01:43:52.455 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-04-23T01:43:52.455 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-23T01:43:52.455 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-23T01:43:52.716 INFO:teuthology.orchestra.run.smithi090.stdout:libcephfs-devel-19.0.0-3192.ga1f34943.el7.x86_6 93 kB/s | 32 kB 00:00 2024-04-23T01:43:52.716 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:43:52.717 INFO:teuthology.orchestra.run.smithi090.stdout:Total 93 kB/s | 32 kB 00:00 2024-04-23T01:43:52.717 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-23T01:43:52.720 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-23T01:43:52.720 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-23T01:43:52.723 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-23T01:43:52.723 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-23T01:43:52.763 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-23T01:43:52.763 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:43:52.763 INFO:teuthology.orchestra.run.smithi113.stdout:Installed: 2024-04-23T01:43:52.763 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-rook-2:19.0.0-3192.ga1f34943.el7.noarch 2024-04-23T01:43:52.763 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-23T01:43:52.763 INFO:teuthology.orchestra.run.smithi113.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-23T01:43:52.763 INFO:teuthology.orchestra.run.smithi113.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-23T01:43:52.763 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-23T01:43:52.764 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-23T01:43:52.764 INFO:teuthology.orchestra.run.smithi113.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-23T01:43:52.764 INFO:teuthology.orchestra.run.smithi113.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-23T01:43:52.764 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-23T01:43:52.764 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-23T01:43:52.764 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-23T01:43:52.764 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-23T01:43:52.764 INFO:teuthology.orchestra.run.smithi113.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-23T01:43:52.764 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:43:52.764 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-23T01:43:52.810 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-23T01:43:52.964 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephfs-devel-2:19.0.0-3192.ga1f34943.el7.x86_64 1/1 2024-04-23T01:43:52.999 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install ceph-mgr-cephadm 2024-04-23T01:43:53.374 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-3192.ga1f34943.el7.x86_64 1/1 2024-04-23T01:43:53.513 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:02:22 ago on Tue 23 Apr 2024 01:41:31 AM UTC. 2024-04-23T01:43:53.588 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephfs-devel-2:19.0.0-3192.ga1f34943.el7.x86_64 1/1 2024-04-23T01:43:53.589 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:53.589 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-23T01:43:53.589 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-devel-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:43:53.589 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:53.589 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-23T01:43:53.633 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-23T01:43:53.634 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:43:53.634 INFO:teuthology.orchestra.run.smithi113.stdout: Package Arch Version Repository Size 2024-04-23T01:43:53.634 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:43:53.634 INFO:teuthology.orchestra.run.smithi113.stdout:Installing: 2024-04-23T01:43:53.634 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3192.ga1f34943.el7 ceph-noarch 142 k 2024-04-23T01:43:53.634 INFO:teuthology.orchestra.run.smithi113.stdout:Installing dependencies: 2024-04-23T01:43:53.634 INFO:teuthology.orchestra.run.smithi113.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-23T01:43:53.634 INFO:teuthology.orchestra.run.smithi113.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-23T01:43:53.634 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-23T01:43:53.634 INFO:teuthology.orchestra.run.smithi113.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-23T01:43:53.635 INFO:teuthology.orchestra.run.smithi113.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-23T01:43:53.635 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:43:53.635 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-04-23T01:43:53.635 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:43:53.635 INFO:teuthology.orchestra.run.smithi113.stdout:Install 6 Packages 2024-04-23T01:43:53.635 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:43:53.635 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 7.0 M 2024-04-23T01:43:53.635 INFO:teuthology.orchestra.run.smithi113.stdout:Installed size: 33 M 2024-04-23T01:43:53.636 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2024-04-23T01:43:53.797 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install librados2 2024-04-23T01:43:54.047 INFO:teuthology.orchestra.run.smithi113.stdout:(1/6): ceph-mgr-cephadm-19.0.0-3192.ga1f34943.e 616 kB/s | 142 kB 00:00 2024-04-23T01:43:54.307 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:02:27 ago on Tue 23 Apr 2024 01:41:27 AM UTC. 2024-04-23T01:43:54.386 INFO:teuthology.orchestra.run.smithi090.stdout:Package librados2-2:19.0.0-3192.ga1f34943.el7.x86_64 is already installed. 2024-04-23T01:43:54.428 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-23T01:43:54.429 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-23T01:43:54.429 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-23T01:43:54.506 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install librbd1 2024-04-23T01:43:54.639 INFO:teuthology.orchestra.run.smithi113.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 926 kB/s | 548 kB 00:00 2024-04-23T01:43:54.797 INFO:teuthology.orchestra.run.smithi113.stdout:(3/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 253 kB/s | 248 kB 00:00 2024-04-23T01:43:54.990 INFO:teuthology.orchestra.run.smithi113.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 165 kB/s | 58 kB 00:00 2024-04-23T01:43:55.013 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:02:28 ago on Tue 23 Apr 2024 01:41:27 AM UTC. 2024-04-23T01:43:55.092 INFO:teuthology.orchestra.run.smithi090.stdout:Package librbd1-2:19.0.0-3192.ga1f34943.el7.x86_64 is already installed. 2024-04-23T01:43:55.133 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-23T01:43:55.134 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-23T01:43:55.134 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-23T01:43:55.140 INFO:teuthology.orchestra.run.smithi113.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 149 kB/s | 51 kB 00:00 2024-04-23T01:43:55.198 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install python3-rados 2024-04-23T01:43:55.712 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:02:28 ago on Tue 23 Apr 2024 01:41:27 AM UTC. 2024-04-23T01:43:55.791 INFO:teuthology.orchestra.run.smithi090.stdout:Package python3-rados-2:19.0.0-3192.ga1f34943.el7.x86_64 is already installed. 2024-04-23T01:43:55.833 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-23T01:43:55.834 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-23T01:43:55.834 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-23T01:43:55.958 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install python3-rgw 2024-04-23T01:43:56.465 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:02:29 ago on Tue 23 Apr 2024 01:41:27 AM UTC. 2024-04-23T01:43:56.543 INFO:teuthology.orchestra.run.smithi090.stdout:Package python3-rgw-2:19.0.0-3192.ga1f34943.el7.x86_64 is already installed. 2024-04-23T01:43:56.584 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-23T01:43:56.585 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-23T01:43:56.586 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-23T01:43:56.668 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install python3-cephfs 2024-04-23T01:43:57.178 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:02:30 ago on Tue 23 Apr 2024 01:41:27 AM UTC. 2024-04-23T01:43:57.257 INFO:teuthology.orchestra.run.smithi090.stdout:Package python3-cephfs-2:19.0.0-3192.ga1f34943.el7.x86_64 is already installed. 2024-04-23T01:43:57.298 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-23T01:43:57.299 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-23T01:43:57.299 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-23T01:43:57.393 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install python3-rbd 2024-04-23T01:43:57.912 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:02:30 ago on Tue 23 Apr 2024 01:41:27 AM UTC. 2024-04-23T01:43:57.991 INFO:teuthology.orchestra.run.smithi090.stdout:Package python3-rbd-2:19.0.0-3192.ga1f34943.el7.x86_64 is already installed. 2024-04-23T01:43:58.033 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-23T01:43:58.034 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-23T01:43:58.034 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-23T01:43:58.101 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install rbd-fuse 2024-04-23T01:43:58.632 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:02:31 ago on Tue 23 Apr 2024 01:41:27 AM UTC. 2024-04-23T01:43:58.771 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-23T01:43:58.772 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:43:58.772 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2024-04-23T01:43:58.772 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:43:58.773 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-23T01:43:58.773 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-fuse x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 90 k 2024-04-23T01:43:58.773 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:58.773 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-23T01:43:58.773 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:43:58.773 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2024-04-23T01:43:58.773 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:43:58.773 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 90 k 2024-04-23T01:43:58.773 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 313 k 2024-04-23T01:43:58.773 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-23T01:43:59.087 INFO:teuthology.orchestra.run.smithi090.stdout:rbd-fuse-19.0.0-3192.ga1f34943.el7.x86_64.rpm 286 kB/s | 90 kB 00:00 2024-04-23T01:43:59.088 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:43:59.089 INFO:teuthology.orchestra.run.smithi090.stdout:Total 284 kB/s | 90 kB 00:00 2024-04-23T01:43:59.089 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-23T01:43:59.098 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-23T01:43:59.098 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-23T01:43:59.164 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-23T01:43:59.164 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-23T01:43:59.312 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-23T01:43:59.419 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-fuse-2:19.0.0-3192.ga1f34943.el7.x86_64 1/1 2024-04-23T01:43:59.907 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: rbd-fuse-2:19.0.0-3192.ga1f34943.el7.x86_64 1/1 2024-04-23T01:44:00.179 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-fuse-2:19.0.0-3192.ga1f34943.el7.x86_64 1/1 2024-04-23T01:44:00.179 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:00.179 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-23T01:44:00.179 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-fuse-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:44:00.179 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:00.179 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-23T01:44:00.448 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install rbd-mirror 2024-04-23T01:44:00.970 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:02:33 ago on Tue 23 Apr 2024 01:41:27 AM UTC. 2024-04-23T01:44:01.093 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-23T01:44:01.093 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:44:01.094 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repo Size 2024-04-23T01:44:01.094 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:44:01.094 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-23T01:44:01.094 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-mirror x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 3.4 M 2024-04-23T01:44:01.094 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:01.094 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-23T01:44:01.094 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:44:01.094 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2024-04-23T01:44:01.094 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:01.094 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 3.4 M 2024-04-23T01:44:01.094 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 29 M 2024-04-23T01:44:01.094 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-23T01:44:01.479 INFO:teuthology.orchestra.run.smithi090.stdout:rbd-mirror-19.0.0-3192.ga1f34943.el7.x86_64.rpm 8.9 MB/s | 3.4 MB 00:00 2024-04-23T01:44:01.480 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:44:01.480 INFO:teuthology.orchestra.run.smithi090.stdout:Total 8.8 MB/s | 3.4 MB 00:00 2024-04-23T01:44:01.480 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-23T01:44:01.491 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-23T01:44:01.491 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-23T01:44:01.574 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-23T01:44:01.574 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-23T01:44:02.132 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-23T01:44:02.238 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-mirror-2:19.0.0-3192.ga1f34943.el7.x86_64 1/1 2024-04-23T01:44:02.239 INFO:teuthology.orchestra.run.smithi113.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 725 kB/s | 6.0 MB 00:08 2024-04-23T01:44:02.240 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:44:02.241 INFO:teuthology.orchestra.run.smithi113.stdout:Total 831 kB/s | 7.0 MB 00:08 2024-04-23T01:44:02.263 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3192.ga1f34943.el7.x86_64 1/1 2024-04-23T01:44:02.263 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T01:44:02.263 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-23T01:44:02.263 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T01:44:02.263 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T01:44:02.263 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:02.315 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2024-04-23T01:44:02.326 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2024-04-23T01:44:02.327 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2024-04-23T01:44:02.423 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2024-04-23T01:44:02.423 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2024-04-23T01:44:02.731 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2024-04-23T01:44:02.912 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-23T01:44:02.965 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-23T01:44:03.476 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-23T01:44:03.568 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-23T01:44:03.571 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-mirror-2:19.0.0-3192.ga1f34943.el7.x86_64 1/1 2024-04-23T01:44:03.571 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:03.571 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-23T01:44:03.571 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-mirror-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:44:03.571 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:03.571 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-23T01:44:03.632 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-23T01:44:03.661 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3192.ga1f34943.el7.noarch 6/6 2024-04-23T01:44:03.858 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install rbd-nbd 2024-04-23T01:44:04.383 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:02:37 ago on Tue 23 Apr 2024 01:41:27 AM UTC. 2024-04-23T01:44:04.472 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3192.ga1f34943.el7.noarch 6/6 2024-04-23T01:44:04.472 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3192.ga1f34943.el7.noarch 1/6 2024-04-23T01:44:04.472 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-23T01:44:04.472 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-23T01:44:04.472 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-23T01:44:04.472 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-23T01:44:04.504 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-23T01:44:04.504 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:44:04.504 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2024-04-23T01:44:04.505 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:44:04.505 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-23T01:44:04.505 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 175 k 2024-04-23T01:44:04.505 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:04.505 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-23T01:44:04.505 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-23T01:44:04.505 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2024-04-23T01:44:04.505 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:04.505 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 175 k 2024-04-23T01:44:04.505 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 673 k 2024-04-23T01:44:04.505 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-23T01:44:04.719 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-23T01:44:04.719 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:04.719 INFO:teuthology.orchestra.run.smithi113.stdout:Installed: 2024-04-23T01:44:04.719 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-cephadm-2:19.0.0-3192.ga1f34943.el7.noarch 2024-04-23T01:44:04.719 INFO:teuthology.orchestra.run.smithi113.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-23T01:44:04.719 INFO:teuthology.orchestra.run.smithi113.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-23T01:44:04.719 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-23T01:44:04.719 INFO:teuthology.orchestra.run.smithi113.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-23T01:44:04.719 INFO:teuthology.orchestra.run.smithi113.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-23T01:44:04.719 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:04.720 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-23T01:44:04.945 INFO:teuthology.orchestra.run.smithi090.stdout:rbd-nbd-19.0.0-3192.ga1f34943.el7.x86_64.rpm 398 kB/s | 175 kB 00:00 2024-04-23T01:44:04.946 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:44:04.946 INFO:teuthology.orchestra.run.smithi090.stdout:Total 397 kB/s | 175 kB 00:00 2024-04-23T01:44:04.946 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-23T01:44:04.956 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-23T01:44:04.956 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-23T01:44:04.969 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install ceph-fuse 2024-04-23T01:44:05.023 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-23T01:44:05.023 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-23T01:44:05.178 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-23T01:44:05.283 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-nbd-2:19.0.0-3192.ga1f34943.el7.x86_64 1/1 2024-04-23T01:44:05.485 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:02:34 ago on Tue 23 Apr 2024 01:41:31 AM UTC. 2024-04-23T01:44:05.605 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-23T01:44:05.606 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:44:05.606 INFO:teuthology.orchestra.run.smithi113.stdout: Package Arch Version Repository Size 2024-04-23T01:44:05.606 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:44:05.606 INFO:teuthology.orchestra.run.smithi113.stdout:Installing: 2024-04-23T01:44:05.606 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-fuse x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 871 k 2024-04-23T01:44:05.606 INFO:teuthology.orchestra.run.smithi113.stdout:Installing dependencies: 2024-04-23T01:44:05.606 INFO:teuthology.orchestra.run.smithi113.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-23T01:44:05.606 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:05.606 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-04-23T01:44:05.606 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:44:05.606 INFO:teuthology.orchestra.run.smithi113.stdout:Install 2 Packages 2024-04-23T01:44:05.606 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:05.607 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 951 k 2024-04-23T01:44:05.607 INFO:teuthology.orchestra.run.smithi113.stdout:Installed size: 4.0 M 2024-04-23T01:44:05.607 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2024-04-23T01:44:05.728 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: rbd-nbd-2:19.0.0-3192.ga1f34943.el7.x86_64 1/1 2024-04-23T01:44:05.958 INFO:teuthology.orchestra.run.smithi113.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 349 kB/s | 80 kB 00:00 2024-04-23T01:44:05.993 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-nbd-2:19.0.0-3192.ga1f34943.el7.x86_64 1/1 2024-04-23T01:44:05.993 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:05.993 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-23T01:44:05.993 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:44:05.993 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:05.993 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-23T01:44:06.150 INFO:teuthology.orchestra.run.smithi113.stdout:(2/2): ceph-fuse-19.0.0-3192.ga1f34943.el7.x86_ 2.0 MB/s | 871 kB 00:00 2024-04-23T01:44:06.151 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:44:06.151 INFO:teuthology.orchestra.run.smithi113.stdout:Total 1.7 MB/s | 951 kB 00:00 2024-04-23T01:44:06.164 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2024-04-23T01:44:06.188 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2024-04-23T01:44:06.188 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2024-04-23T01:44:06.265 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2024-04-23T01:44:06.266 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2024-04-23T01:44:06.292 DEBUG:teuthology.parallel:result is None 2024-04-23T01:44:06.450 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2024-04-23T01:44:06.584 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-23T01:44:06.649 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-fuse-2:19.0.0-3192.ga1f34943.el7.x86_64 2/2 2024-04-23T01:44:07.306 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-fuse-2:19.0.0-3192.ga1f34943.el7.x86_64 2/2 2024-04-23T01:44:07.306 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-fuse-2:19.0.0-3192.ga1f34943.el7.x86_64 1/2 2024-04-23T01:44:07.577 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-23T01:44:07.577 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:07.577 INFO:teuthology.orchestra.run.smithi113.stdout:Installed: 2024-04-23T01:44:07.577 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-fuse-2:19.0.0-3192.ga1f34943.el7.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-23T01:44:07.578 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:07.578 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-23T01:44:07.862 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install ceph-volume 2024-04-23T01:44:08.382 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:02:37 ago on Tue 23 Apr 2024 01:41:31 AM UTC. 2024-04-23T01:44:08.504 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-23T01:44:08.505 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:44:08.505 INFO:teuthology.orchestra.run.smithi113.stdout: Package Arch Version Repository Size 2024-04-23T01:44:08.505 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:44:08.505 INFO:teuthology.orchestra.run.smithi113.stdout:Installing: 2024-04-23T01:44:08.505 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-volume noarch 2:19.0.0-3192.ga1f34943.el7 ceph-noarch 273 k 2024-04-23T01:44:08.505 INFO:teuthology.orchestra.run.smithi113.stdout:Installing dependencies: 2024-04-23T01:44:08.505 INFO:teuthology.orchestra.run.smithi113.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-23T01:44:08.505 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:08.506 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-04-23T01:44:08.506 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:44:08.506 INFO:teuthology.orchestra.run.smithi113.stdout:Install 2 Packages 2024-04-23T01:44:08.506 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:08.506 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 350 k 2024-04-23T01:44:08.506 INFO:teuthology.orchestra.run.smithi113.stdout:Installed size: 1.5 M 2024-04-23T01:44:08.507 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2024-04-23T01:44:08.923 INFO:teuthology.orchestra.run.smithi113.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 545 kB/s | 77 kB 00:00 2024-04-23T01:44:09.048 INFO:teuthology.orchestra.run.smithi113.stdout:(2/2): ceph-volume-19.0.0-3192.ga1f34943.el7.no 1.0 MB/s | 273 kB 00:00 2024-04-23T01:44:09.048 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:44:09.049 INFO:teuthology.orchestra.run.smithi113.stdout:Total 645 kB/s | 350 kB 00:00 2024-04-23T01:44:09.060 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2024-04-23T01:44:09.069 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2024-04-23T01:44:09.070 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2024-04-23T01:44:09.095 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2024-04-23T01:44:09.096 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2024-04-23T01:44:09.235 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2024-04-23T01:44:09.429 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-04-23T01:44:09.467 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-volume-2:19.0.0-3192.ga1f34943.el7.noarch 2/2 2024-04-23T01:44:09.482 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-volume-2:19.0.0-3192.ga1f34943.el7.noarch 2/2 2024-04-23T01:44:09.482 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T01:44:09.482 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-23T01:44:09.483 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:10.190 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-volume-2:19.0.0-3192.ga1f34943.el7.noarch 1/2 2024-04-23T01:44:10.476 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-04-23T01:44:10.476 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:10.476 INFO:teuthology.orchestra.run.smithi113.stdout:Installed: 2024-04-23T01:44:10.476 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-volume-2:19.0.0-3192.ga1f34943.el7.noarch 2024-04-23T01:44:10.476 INFO:teuthology.orchestra.run.smithi113.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-23T01:44:10.476 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:10.476 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-23T01:44:10.810 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install librados-devel 2024-04-23T01:44:11.323 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:02:40 ago on Tue 23 Apr 2024 01:41:31 AM UTC. 2024-04-23T01:44:11.445 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-23T01:44:11.446 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:44:11.446 INFO:teuthology.orchestra.run.smithi113.stdout: Package Arch Version Repo Size 2024-04-23T01:44:11.446 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:44:11.446 INFO:teuthology.orchestra.run.smithi113.stdout:Installing: 2024-04-23T01:44:11.446 INFO:teuthology.orchestra.run.smithi113.stdout: librados-devel x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 125 k 2024-04-23T01:44:11.446 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:11.446 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-04-23T01:44:11.446 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:44:11.446 INFO:teuthology.orchestra.run.smithi113.stdout:Install 1 Package 2024-04-23T01:44:11.446 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:11.447 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 125 k 2024-04-23T01:44:11.447 INFO:teuthology.orchestra.run.smithi113.stdout:Installed size: 445 k 2024-04-23T01:44:11.447 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2024-04-23T01:44:12.022 INFO:teuthology.orchestra.run.smithi113.stdout:librados-devel-19.0.0-3192.ga1f34943.el7.x86_64 217 kB/s | 125 kB 00:00 2024-04-23T01:44:12.023 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:44:12.023 INFO:teuthology.orchestra.run.smithi113.stdout:Total 217 kB/s | 125 kB 00:00 2024-04-23T01:44:12.023 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2024-04-23T01:44:12.028 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2024-04-23T01:44:12.028 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2024-04-23T01:44:12.091 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2024-04-23T01:44:12.092 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2024-04-23T01:44:12.250 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2024-04-23T01:44:12.377 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librados-devel-2:19.0.0-3192.ga1f34943.el7.x86_64 1/1 2024-04-23T01:44:12.787 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librados-devel-2:19.0.0-3192.ga1f34943.el7.x86_64 1/1 2024-04-23T01:44:13.046 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados-devel-2:19.0.0-3192.ga1f34943.el7.x86_64 1/1 2024-04-23T01:44:13.046 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:13.046 INFO:teuthology.orchestra.run.smithi113.stdout:Installed: 2024-04-23T01:44:13.046 INFO:teuthology.orchestra.run.smithi113.stdout: librados-devel-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:44:13.046 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:13.046 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-23T01:44:13.315 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install libcephfs2 2024-04-23T01:44:13.836 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:02:42 ago on Tue 23 Apr 2024 01:41:31 AM UTC. 2024-04-23T01:44:13.915 INFO:teuthology.orchestra.run.smithi113.stdout:Package libcephfs2-2:19.0.0-3192.ga1f34943.el7.x86_64 is already installed. 2024-04-23T01:44:13.956 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-23T01:44:13.958 INFO:teuthology.orchestra.run.smithi113.stdout:Nothing to do. 2024-04-23T01:44:13.958 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-23T01:44:14.045 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install libcephfs-devel 2024-04-23T01:44:14.556 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:02:43 ago on Tue 23 Apr 2024 01:41:31 AM UTC. 2024-04-23T01:44:14.675 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-23T01:44:14.676 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:44:14.676 INFO:teuthology.orchestra.run.smithi113.stdout: Package Arch Version Repo Size 2024-04-23T01:44:14.676 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:44:14.676 INFO:teuthology.orchestra.run.smithi113.stdout:Installing: 2024-04-23T01:44:14.676 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs-devel x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 32 k 2024-04-23T01:44:14.676 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:14.676 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-04-23T01:44:14.676 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:44:14.676 INFO:teuthology.orchestra.run.smithi113.stdout:Install 1 Package 2024-04-23T01:44:14.676 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:14.677 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 32 k 2024-04-23T01:44:14.677 INFO:teuthology.orchestra.run.smithi113.stdout:Installed size: 142 k 2024-04-23T01:44:14.677 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2024-04-23T01:44:14.994 INFO:teuthology.orchestra.run.smithi113.stdout:libcephfs-devel-19.0.0-3192.ga1f34943.el7.x86_6 100 kB/s | 32 kB 00:00 2024-04-23T01:44:14.995 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:44:14.995 INFO:teuthology.orchestra.run.smithi113.stdout:Total 99 kB/s | 32 kB 00:00 2024-04-23T01:44:14.995 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2024-04-23T01:44:14.998 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2024-04-23T01:44:14.999 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2024-04-23T01:44:15.001 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2024-04-23T01:44:15.001 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2024-04-23T01:44:15.088 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2024-04-23T01:44:15.189 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephfs-devel-2:19.0.0-3192.ga1f34943.el7.x86_64 1/1 2024-04-23T01:44:15.548 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-3192.ga1f34943.el7.x86_64 1/1 2024-04-23T01:44:15.768 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephfs-devel-2:19.0.0-3192.ga1f34943.el7.x86_64 1/1 2024-04-23T01:44:15.768 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:15.768 INFO:teuthology.orchestra.run.smithi113.stdout:Installed: 2024-04-23T01:44:15.768 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs-devel-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:44:15.768 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:15.768 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-23T01:44:15.952 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install librados2 2024-04-23T01:44:16.489 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:02:45 ago on Tue 23 Apr 2024 01:41:31 AM UTC. 2024-04-23T01:44:16.569 INFO:teuthology.orchestra.run.smithi113.stdout:Package librados2-2:19.0.0-3192.ga1f34943.el7.x86_64 is already installed. 2024-04-23T01:44:16.611 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-23T01:44:16.613 INFO:teuthology.orchestra.run.smithi113.stdout:Nothing to do. 2024-04-23T01:44:16.613 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-23T01:44:16.682 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install librbd1 2024-04-23T01:44:17.196 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:02:46 ago on Tue 23 Apr 2024 01:41:31 AM UTC. 2024-04-23T01:44:17.274 INFO:teuthology.orchestra.run.smithi113.stdout:Package librbd1-2:19.0.0-3192.ga1f34943.el7.x86_64 is already installed. 2024-04-23T01:44:17.316 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-23T01:44:17.317 INFO:teuthology.orchestra.run.smithi113.stdout:Nothing to do. 2024-04-23T01:44:17.317 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-23T01:44:17.382 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install python3-rados 2024-04-23T01:44:17.893 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:02:46 ago on Tue 23 Apr 2024 01:41:31 AM UTC. 2024-04-23T01:44:17.970 INFO:teuthology.orchestra.run.smithi113.stdout:Package python3-rados-2:19.0.0-3192.ga1f34943.el7.x86_64 is already installed. 2024-04-23T01:44:18.011 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-23T01:44:18.012 INFO:teuthology.orchestra.run.smithi113.stdout:Nothing to do. 2024-04-23T01:44:18.012 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-23T01:44:18.085 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install python3-rgw 2024-04-23T01:44:18.602 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:02:47 ago on Tue 23 Apr 2024 01:41:31 AM UTC. 2024-04-23T01:44:18.680 INFO:teuthology.orchestra.run.smithi113.stdout:Package python3-rgw-2:19.0.0-3192.ga1f34943.el7.x86_64 is already installed. 2024-04-23T01:44:18.721 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-23T01:44:18.722 INFO:teuthology.orchestra.run.smithi113.stdout:Nothing to do. 2024-04-23T01:44:18.722 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-23T01:44:18.797 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install python3-cephfs 2024-04-23T01:44:19.304 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:02:48 ago on Tue 23 Apr 2024 01:41:31 AM UTC. 2024-04-23T01:44:19.382 INFO:teuthology.orchestra.run.smithi113.stdout:Package python3-cephfs-2:19.0.0-3192.ga1f34943.el7.x86_64 is already installed. 2024-04-23T01:44:19.424 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-23T01:44:19.425 INFO:teuthology.orchestra.run.smithi113.stdout:Nothing to do. 2024-04-23T01:44:19.425 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-23T01:44:19.498 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install python3-rbd 2024-04-23T01:44:20.007 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:02:49 ago on Tue 23 Apr 2024 01:41:31 AM UTC. 2024-04-23T01:44:20.085 INFO:teuthology.orchestra.run.smithi113.stdout:Package python3-rbd-2:19.0.0-3192.ga1f34943.el7.x86_64 is already installed. 2024-04-23T01:44:20.126 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-23T01:44:20.127 INFO:teuthology.orchestra.run.smithi113.stdout:Nothing to do. 2024-04-23T01:44:20.127 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-23T01:44:20.207 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install rbd-fuse 2024-04-23T01:44:20.720 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:02:49 ago on Tue 23 Apr 2024 01:41:31 AM UTC. 2024-04-23T01:44:20.842 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-23T01:44:20.842 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:44:20.842 INFO:teuthology.orchestra.run.smithi113.stdout: Package Architecture Version Repository Size 2024-04-23T01:44:20.842 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:44:20.842 INFO:teuthology.orchestra.run.smithi113.stdout:Installing: 2024-04-23T01:44:20.842 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-fuse x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 90 k 2024-04-23T01:44:20.842 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:20.842 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-04-23T01:44:20.843 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:44:20.843 INFO:teuthology.orchestra.run.smithi113.stdout:Install 1 Package 2024-04-23T01:44:20.843 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:20.843 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 90 k 2024-04-23T01:44:20.843 INFO:teuthology.orchestra.run.smithi113.stdout:Installed size: 313 k 2024-04-23T01:44:20.843 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2024-04-23T01:44:21.048 INFO:teuthology.orchestra.run.smithi113.stdout:rbd-fuse-19.0.0-3192.ga1f34943.el7.x86_64.rpm 436 kB/s | 90 kB 00:00 2024-04-23T01:44:21.049 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:44:21.049 INFO:teuthology.orchestra.run.smithi113.stdout:Total 433 kB/s | 90 kB 00:00 2024-04-23T01:44:21.049 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2024-04-23T01:44:21.058 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2024-04-23T01:44:21.058 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2024-04-23T01:44:21.120 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2024-04-23T01:44:21.120 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2024-04-23T01:44:21.271 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2024-04-23T01:44:21.369 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-fuse-2:19.0.0-3192.ga1f34943.el7.x86_64 1/1 2024-04-23T01:44:21.736 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: rbd-fuse-2:19.0.0-3192.ga1f34943.el7.x86_64 1/1 2024-04-23T01:44:21.973 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-fuse-2:19.0.0-3192.ga1f34943.el7.x86_64 1/1 2024-04-23T01:44:21.973 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:21.973 INFO:teuthology.orchestra.run.smithi113.stdout:Installed: 2024-04-23T01:44:21.973 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-fuse-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:44:21.973 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:21.973 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-23T01:44:22.208 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install rbd-mirror 2024-04-23T01:44:22.727 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:02:51 ago on Tue 23 Apr 2024 01:41:31 AM UTC. 2024-04-23T01:44:22.849 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-23T01:44:22.849 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:44:22.849 INFO:teuthology.orchestra.run.smithi113.stdout: Package Arch Version Repo Size 2024-04-23T01:44:22.850 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:44:22.850 INFO:teuthology.orchestra.run.smithi113.stdout:Installing: 2024-04-23T01:44:22.850 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-mirror x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 3.4 M 2024-04-23T01:44:22.850 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:22.850 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-04-23T01:44:22.850 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:44:22.850 INFO:teuthology.orchestra.run.smithi113.stdout:Install 1 Package 2024-04-23T01:44:22.850 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:22.850 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 3.4 M 2024-04-23T01:44:22.850 INFO:teuthology.orchestra.run.smithi113.stdout:Installed size: 29 M 2024-04-23T01:44:22.850 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2024-04-23T01:44:23.736 INFO:teuthology.orchestra.run.smithi113.stdout:rbd-mirror-19.0.0-3192.ga1f34943.el7.x86_64.rpm 3.8 MB/s | 3.4 MB 00:00 2024-04-23T01:44:23.737 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:44:23.738 INFO:teuthology.orchestra.run.smithi113.stdout:Total 3.8 MB/s | 3.4 MB 00:00 2024-04-23T01:44:23.738 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2024-04-23T01:44:23.749 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2024-04-23T01:44:23.749 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2024-04-23T01:44:23.831 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2024-04-23T01:44:23.831 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2024-04-23T01:44:24.393 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2024-04-23T01:44:24.462 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-mirror-2:19.0.0-3192.ga1f34943.el7.x86_64 1/1 2024-04-23T01:44:24.487 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3192.ga1f34943.el7.x86_64 1/1 2024-04-23T01:44:24.487 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T01:44:24.487 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-23T01:44:24.487 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T01:44:24.487 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T01:44:24.487 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:25.669 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-mirror-2:19.0.0-3192.ga1f34943.el7.x86_64 1/1 2024-04-23T01:44:25.670 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:25.670 INFO:teuthology.orchestra.run.smithi113.stdout:Installed: 2024-04-23T01:44:25.670 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-mirror-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:44:25.670 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:25.670 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-23T01:44:25.927 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install rbd-nbd 2024-04-23T01:44:26.448 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:02:55 ago on Tue 23 Apr 2024 01:41:31 AM UTC. 2024-04-23T01:44:26.584 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-23T01:44:26.585 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:44:26.585 INFO:teuthology.orchestra.run.smithi113.stdout: Package Architecture Version Repository Size 2024-04-23T01:44:26.585 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:44:26.585 INFO:teuthology.orchestra.run.smithi113.stdout:Installing: 2024-04-23T01:44:26.585 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-nbd x86_64 2:19.0.0-3192.ga1f34943.el7 ceph 175 k 2024-04-23T01:44:26.585 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:26.585 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-04-23T01:44:26.585 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-23T01:44:26.585 INFO:teuthology.orchestra.run.smithi113.stdout:Install 1 Package 2024-04-23T01:44:26.585 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:26.586 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 175 k 2024-04-23T01:44:26.586 INFO:teuthology.orchestra.run.smithi113.stdout:Installed size: 673 k 2024-04-23T01:44:26.586 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2024-04-23T01:44:27.049 INFO:teuthology.orchestra.run.smithi113.stdout:rbd-nbd-19.0.0-3192.ga1f34943.el7.x86_64.rpm 378 kB/s | 175 kB 00:00 2024-04-23T01:44:27.050 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:44:27.050 INFO:teuthology.orchestra.run.smithi113.stdout:Total 377 kB/s | 175 kB 00:00 2024-04-23T01:44:27.050 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2024-04-23T01:44:27.060 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2024-04-23T01:44:27.060 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2024-04-23T01:44:27.128 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2024-04-23T01:44:27.128 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2024-04-23T01:44:27.281 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2024-04-23T01:44:27.378 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-nbd-2:19.0.0-3192.ga1f34943.el7.x86_64 1/1 2024-04-23T01:44:27.771 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: rbd-nbd-2:19.0.0-3192.ga1f34943.el7.x86_64 1/1 2024-04-23T01:44:27.993 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-nbd-2:19.0.0-3192.ga1f34943.el7.x86_64 1/1 2024-04-23T01:44:27.994 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:27.994 INFO:teuthology.orchestra.run.smithi113.stdout:Installed: 2024-04-23T01:44:27.994 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-nbd-2:19.0.0-3192.ga1f34943.el7.x86_64 2024-04-23T01:44:27.994 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:27.994 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-23T01:44:28.236 DEBUG:teuthology.parallel:result is None 2024-04-23T01:44:28.236 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4 2024-04-23T01:44:28.462 DEBUG:teuthology.orchestra.run.smithi090:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-23T01:44:28.487 INFO:teuthology.orchestra.run.smithi090.stdout:19.0.0-3192.ga1f34943.el7 2024-04-23T01:44:28.487 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3192.ga1f34943.el7 2024-04-23T01:44:28.487 INFO:teuthology.task.install:The correct ceph version 19.0.0-3192.ga1f34943 is installed. 2024-04-23T01:44:28.489 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4 2024-04-23T01:44:28.633 DEBUG:teuthology.orchestra.run.smithi113:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-23T01:44:28.654 INFO:teuthology.orchestra.run.smithi113.stdout:19.0.0-3192.ga1f34943.el7 2024-04-23T01:44:28.655 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3192.ga1f34943.el7 2024-04-23T01:44:28.655 INFO:teuthology.task.install:The correct ceph version 19.0.0-3192.ga1f34943 is installed. 2024-04-23T01:44:28.656 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-23T01:44:28.656 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-23T01:44:28.656 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T01:44:28.685 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-23T01:44:28.685 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T01:44:28.719 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-23T01:44:28.720 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-23T01:44:28.720 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T01:44:28.752 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T01:44:28.818 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-23T01:44:28.818 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T01:44:28.843 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T01:44:28.907 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-23T01:44:28.907 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-23T01:44:28.907 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T01:44:28.933 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T01:44:28.998 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-23T01:44:28.998 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T01:44:29.021 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T01:44:29.085 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-23T01:44:29.085 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-23T01:44:29.085 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T01:44:29.108 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T01:44:29.172 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-23T01:44:29.172 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T01:44:29.195 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T01:44:29.258 INFO:teuthology.run_tasks:Running task ceph... 2024-04-23T01:44:29.338 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-23T01:44:29.338 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 777 /var/log/ceph 2024-04-23T01:44:29.340 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 777 /var/log/ceph 2024-04-23T01:44:29.372 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-23T01:44:29.372 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-23T01:44:29.414 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-23T01:44:29.437 INFO:tasks.ceph:Creating extra log directories... 2024-04-23T01:44:29.437 DEBUG:teuthology.orchestra.run.smithi090:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-23T01:44:29.477 DEBUG:teuthology.orchestra.run.smithi113:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-23T01:44:29.509 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-23T01:44:29.509 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore rocksdb cf': False, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'err to stderr': True, 'log flush on exit': True, 'log to file': False, 'log to stderr': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-23T01:44:29.510 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yingxin-2024-04-23_01:23:55-crimson-rados-ci-yingxin-crimson-fix-tri-mutex-4-distro-default-smithi/7669459', 'branch': 'ci-yingxin-crimson-fix-tri-mutex-4', 'description': 'crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/simple thrashosds-health workloads/radosbench-high-concurrency}', 'email': 'yingxin.cheng@intel.com', 'first_in_suite': False, 'job_id': '7669459', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yingxin-2024-04-23_01:23:55-crimson-rados-ci-yingxin-crimson-fix-tri-mutex-4-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'ci-yingxin-crimson-fix-tri-mutex-4'}, 'ceph': {'conf': {'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore rocksdb cf': False, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'err to stderr': True, 'log flush on exit': True, 'log to file': False, 'log to stderr': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'flavor': 'crimson', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum'], 'sha1': 'a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'crimson', 'sha1': 'a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'min_in': 4}, 'workunit': {'branch': 'ci-yingxin-crimson-fix-tri-mutex-4', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': 'a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4'}}, 'owner': 'scheduled_yingxin@teuthology', 'priority': 60, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'], ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b']], 'sha1': 'a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'ci-yingxin-crimson-fix-tri-mutex-4', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4', 'targets': {'smithi090.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAWC5iJNafd1R+E2MWb4IxaYKCxUJ+HluqBuYCuGUXLeh2aLSeliTX1hcdEivTGrHmcBSK7uu5wXYfheVEoQH4I=', 'smithi113.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCKdEn9EAfV/3De7+BKt/+rJPlg1u5r1QRzsikh2mPtC8ERZ/gQkM/mGeLMZrXk30QriId993Gxm6p+dUkuqAB4='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore rocksdb cf': False, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'err to stderr': True, 'log flush on exit': True, 'log to file': False, 'log to stderr': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'flavor': 'crimson', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum'], 'sha1': 'a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4', 'cluster': 'ceph'}}, {'thrashosds': {'ceph_objectstore_tool': False, 'chance_down': 0, 'chance_inject_pause_short': 0, 'chance_reset_purged_snaps_last': 0, 'chance_thrash_cluster_full': 0, 'chance_thrash_pg_upmap': 0, 'dump_ops_enable': False, 'min_in': 3, 'noscrub_toggle_delay': 0, 'reweight_osd': 0, 'sighup_delay': 0, 'thrash_primary_affinity': False, 'timeout': 2400}}, {'full_sequential': [{'radosbench': {'clients': ['client.0'], 'concurrency': 128, 'size': 8192, 'time': 90}}, {'radosbench': {'clients': ['client.0'], 'concurrency': 128, 'size': 8192, 'time': 90}}, {'radosbench': {'clients': ['client.0'], 'concurrency': 128, 'size': 8192, 'time': 90}}, {'radosbench': {'clients': ['client.0'], 'concurrency': 128, 'size': 8192, 'time': 90}}, {'radosbench': {'clients': ['client.0'], 'concurrency': 128, 'size': 8192, 'time': 90}}, {'radosbench': {'clients': ['client.0'], 'concurrency': 128, 'size': 8192, 'time': 90}}, {'radosbench': {'clients': ['client.0'], 'concurrency': 128, 'size': 8192, 'time': 90}}, {'radosbench': {'clients': ['client.0'], 'concurrency': 128, 'size': 8192, 'time': 90}}]}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-23_01:23:55', 'tube': 'smithi', 'user': 'yingxin', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-23T01:44:29.510 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-23T01:44:29.541 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-23T01:44:29.556 DEBUG:teuthology.orchestra.run.smithi090:> sudo install -d -m0777 -- /var/run/ceph 2024-04-23T01:44:29.596 DEBUG:teuthology.orchestra.run.smithi113:> sudo install -d -m0777 -- /var/run/ceph 2024-04-23T01:44:29.619 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-23T01:44:29.619 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T01:44:29.672 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T01:44:29.672 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_1 2024-04-23T01:44:29.726 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T01:44:29.726 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T01:44:29.726 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-23T01:44:29.726 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T01:44:29.727 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T01:44:29.727 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-04-23 01:44:02.534292378 +0000 2024-04-23T01:44:29.727 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-04-23 01:40:36.013696342 +0000 2024-04-23T01:44:29.727 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-04-23 01:40:36.013696342 +0000 2024-04-23T01:44:29.727 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-04-23 01:40:36.013696342 +0000 2024-04-23T01:44:29.727 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T01:44:29.787 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-04-23T01:44:29.787 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-04-23T01:44:29.788 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.00023549 s, 2.2 MB/s 2024-04-23T01:44:29.789 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T01:44:29.844 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_2 2024-04-23T01:44:29.898 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T01:44:29.898 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T01:44:29.898 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-04-23T01:44:29.898 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T01:44:29.898 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T01:44:29.898 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-04-23 01:44:02.534292378 +0000 2024-04-23T01:44:29.898 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-04-23 01:40:36.350691156 +0000 2024-04-23T01:44:29.899 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-04-23 01:40:36.350691156 +0000 2024-04-23T01:44:29.899 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-04-23 01:40:36.350691156 +0000 2024-04-23T01:44:29.899 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T01:44:29.959 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-04-23T01:44:29.959 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-04-23T01:44:29.959 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000168998 s, 3.0 MB/s 2024-04-23T01:44:29.961 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T01:44:30.016 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_3 2024-04-23T01:44:30.070 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T01:44:30.070 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T01:44:30.070 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-23T01:44:30.071 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T01:44:30.071 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T01:44:30.071 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-04-23 01:44:02.535292363 +0000 2024-04-23T01:44:30.071 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-04-23 01:40:36.674686170 +0000 2024-04-23T01:44:30.071 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-04-23 01:40:36.674686170 +0000 2024-04-23T01:44:30.071 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-04-23 01:40:36.674686170 +0000 2024-04-23T01:44:30.071 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T01:44:30.132 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-04-23T01:44:30.132 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-04-23T01:44:30.132 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000178179 s, 2.9 MB/s 2024-04-23T01:44:30.133 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T01:44:30.189 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_4 2024-04-23T01:44:30.244 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T01:44:30.245 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T01:44:30.245 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-23T01:44:30.245 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T01:44:30.245 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T01:44:30.245 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-04-23 01:44:02.535292363 +0000 2024-04-23T01:44:30.245 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-04-23 01:40:36.996681216 +0000 2024-04-23T01:44:30.245 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-04-23 01:40:36.996681216 +0000 2024-04-23T01:44:30.245 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-04-23 01:40:36.996681216 +0000 2024-04-23T01:44:30.245 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T01:44:30.307 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-04-23T01:44:30.307 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-04-23T01:44:30.308 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.0001714 s, 3.0 MB/s 2024-04-23T01:44:30.309 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T01:44:30.364 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-04-23T01:44:30.364 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-23T01:44:30.364 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T01:44:30.380 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T01:44:30.381 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_1 2024-04-23T01:44:30.436 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T01:44:30.436 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T01:44:30.436 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-23T01:44:30.436 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T01:44:30.436 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T01:44:30.436 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-04-23 01:44:24.740375410 +0000 2024-04-23T01:44:30.436 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-04-23 01:40:36.009654652 +0000 2024-04-23T01:44:30.436 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-04-23 01:40:36.009654652 +0000 2024-04-23T01:44:30.436 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-04-23 01:40:36.009654652 +0000 2024-04-23T01:44:30.436 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T01:44:30.499 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-23T01:44:30.499 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-23T01:44:30.499 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000214943 s, 2.4 MB/s 2024-04-23T01:44:30.500 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T01:44:30.556 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_2 2024-04-23T01:44:30.611 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T01:44:30.611 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T01:44:30.611 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-23T01:44:30.611 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T01:44:30.611 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T01:44:30.612 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-04-23 01:44:24.740375410 +0000 2024-04-23T01:44:30.612 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-04-23 01:40:36.346647320 +0000 2024-04-23T01:44:30.612 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-04-23 01:40:36.346647320 +0000 2024-04-23T01:44:30.612 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-04-23 01:40:36.346647320 +0000 2024-04-23T01:44:30.612 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T01:44:30.673 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-23T01:44:30.673 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-23T01:44:30.673 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000197866 s, 2.6 MB/s 2024-04-23T01:44:30.674 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T01:44:30.729 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_3 2024-04-23T01:44:30.784 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T01:44:30.784 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T01:44:30.784 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-23T01:44:30.784 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T01:44:30.784 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T01:44:30.784 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-04-23 01:44:24.741375389 +0000 2024-04-23T01:44:30.784 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-04-23 01:40:36.696639705 +0000 2024-04-23T01:44:30.784 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-04-23 01:40:36.696639705 +0000 2024-04-23T01:44:30.784 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-04-23 01:40:36.696639705 +0000 2024-04-23T01:44:30.784 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T01:44:30.845 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-23T01:44:30.845 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-23T01:44:30.845 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000189799 s, 2.7 MB/s 2024-04-23T01:44:30.847 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T01:44:30.902 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_4 2024-04-23T01:44:30.957 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T01:44:30.957 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T01:44:30.957 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-23T01:44:30.957 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T01:44:30.957 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T01:44:30.957 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-04-23 01:44:24.741375389 +0000 2024-04-23T01:44:30.957 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-04-23 01:40:37.024632568 +0000 2024-04-23T01:44:30.957 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-04-23 01:40:37.024632568 +0000 2024-04-23T01:44:30.957 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-04-23 01:40:37.024632568 +0000 2024-04-23T01:44:30.957 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T01:44:31.019 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-23T01:44:31.068 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-23T01:44:31.069 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000231466 s, 2.2 MB/s 2024-04-23T01:44:31.070 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T01:44:31.125 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-04-23T01:44:31.126 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi090.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi113.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2024-04-23T01:44:31.126 INFO:tasks.ceph:Generating config... 2024-04-23T01:44:31.127 INFO:tasks.ceph:[osd] debug monc = 20 2024-04-23T01:44:31.127 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-23T01:44:31.127 INFO:tasks.ceph:[osd] bluestore compression algorithm = snappy 2024-04-23T01:44:31.127 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-04-23T01:44:31.127 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-23T01:44:31.127 INFO:tasks.ceph:[osd] bluestore rocksdb cf = False 2024-04-23T01:44:31.127 INFO:tasks.ceph:[osd] crimson alien thread cpu cores = 6-7 2024-04-23T01:44:31.127 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2024-04-23T01:44:31.127 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-23T01:44:31.127 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-23T01:44:31.127 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-23T01:44:31.128 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-23T01:44:31.128 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-23T01:44:31.128 INFO:tasks.ceph:[osd] err to stderr = True 2024-04-23T01:44:31.128 INFO:tasks.ceph:[osd] log flush on exit = True 2024-04-23T01:44:31.128 INFO:tasks.ceph:[osd] log to file = False 2024-04-23T01:44:31.128 INFO:tasks.ceph:[osd] log to stderr = True 2024-04-23T01:44:31.128 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-23T01:44:31.128 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-23T01:44:31.128 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-23T01:44:31.128 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-04-23T01:44:31.128 INFO:tasks.ceph:[osd] osd debug reject backfill probability = 0.3 2024-04-23T01:44:31.128 INFO:tasks.ceph:[osd] osd delete sleep = 1 2024-04-23T01:44:31.128 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-23T01:44:31.128 INFO:tasks.ceph:[osd] osd max backfills = 3 2024-04-23T01:44:31.128 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-04-23T01:44:31.128 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-23T01:44:31.129 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2024-04-23T01:44:31.129 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2024-04-23T01:44:31.129 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-04-23T01:44:31.129 INFO:tasks.ceph:[osd] osd snap trim sleep = 2 2024-04-23T01:44:31.129 INFO:tasks.ceph:[client.0] debug ms = 1 2024-04-23T01:44:31.129 INFO:tasks.ceph:[client.0] debug objecter = 20 2024-04-23T01:44:31.129 INFO:tasks.ceph:[client.0] debug rados = 20 2024-04-23T01:44:31.129 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2024-04-23T01:44:31.129 INFO:tasks.ceph:[global] ms client mode = crc 2024-04-23T01:44:31.129 INFO:tasks.ceph:[global] ms cluster mode = crc 2024-04-23T01:44:31.129 INFO:tasks.ceph:[global] ms mon client mode = crc 2024-04-23T01:44:31.129 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2024-04-23T01:44:31.129 INFO:tasks.ceph:[global] ms mon service mode = crc 2024-04-23T01:44:31.129 INFO:tasks.ceph:[global] ms service mode = crc 2024-04-23T01:44:31.129 INFO:tasks.ceph:[global] osd_pool_default_min_size = 2 2024-04-23T01:44:31.129 INFO:tasks.ceph:[global] osd_pool_default_size = 3 2024-04-23T01:44:31.129 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-23T01:44:31.130 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-23T01:44:31.130 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-23T01:44:31.130 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-23T01:44:31.130 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-23T01:44:31.130 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2024-04-23T01:44:31.130 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-04-23T01:44:31.130 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-04-23T01:44:31.130 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-04-23T01:44:31.130 INFO:tasks.ceph:[mon] osd pool default crimson = True 2024-04-23T01:44:31.130 INFO:tasks.ceph:[mon] paxos service trim min = 10 2024-04-23T01:44:31.130 INFO:tasks.ceph:[osd.0] crimson seastar cpu cores = 0-2 2024-04-23T01:44:31.130 INFO:tasks.ceph:[osd.1] crimson seastar cpu cores = 3-5 2024-04-23T01:44:31.130 INFO:tasks.ceph:[osd.2] crimson seastar cpu cores = 0-2 2024-04-23T01:44:31.131 INFO:tasks.ceph:[osd.3] crimson seastar cpu cores = 3-5 2024-04-23T01:44:31.131 INFO:tasks.ceph:Setting up mon.a... 2024-04-23T01:44:31.131 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-23T01:44:31.218 INFO:teuthology.orchestra.run.smithi090.stdout:creating /etc/ceph/ceph.keyring 2024-04-23T01:44:31.221 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-23T01:44:31.272 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-23T01:44:31.295 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.90')] 2024-04-23T01:44:31.296 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.90', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore rocksdb cf': False, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'err to stderr': True, 'log flush on exit': True, 'log to file': False, 'log to stderr': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}} 2024-04-23T01:44:31.296 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-23T01:44:31.296 DEBUG:teuthology.orchestra.run.smithi090:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-23T01:44:31.350 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.90 --print /home/ubuntu/cephtest/ceph.monmap 2024-04-23T01:44:31.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:44:31.441+0000 7efce702bf00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:44:31.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:44:31.441+0000 7efce702bf00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:44:31.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:44:31.442+0000 7efce702bf00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:44:31.441 INFO:teuthology.orchestra.run.smithi090.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-23T01:44:31.441 INFO:teuthology.orchestra.run.smithi090.stdout:monmaptool: generated fsid 0a731bfd-0fc4-4c1b-8e4b-f6dfbc059bcc 2024-04-23T01:44:31.441 INFO:teuthology.orchestra.run.smithi090.stdout:setting min_mon_release = quincy 2024-04-23T01:44:31.441 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 0 2024-04-23T01:44:31.441 INFO:teuthology.orchestra.run.smithi090.stdout:fsid 0a731bfd-0fc4-4c1b-8e4b-f6dfbc059bcc 2024-04-23T01:44:31.441 INFO:teuthology.orchestra.run.smithi090.stdout:last_changed 2024-04-23T01:44:31.443383+0000 2024-04-23T01:44:31.441 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-23T01:44:31.443383+0000 2024-04-23T01:44:31.441 INFO:teuthology.orchestra.run.smithi090.stdout:min_mon_release 17 (quincy) 2024-04-23T01:44:31.441 INFO:teuthology.orchestra.run.smithi090.stdout:election_strategy: 1 2024-04-23T01:44:31.441 INFO:teuthology.orchestra.run.smithi090.stdout:0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-04-23T01:44:31.442 INFO:teuthology.orchestra.run.smithi090.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2024-04-23T01:44:31.443 DEBUG:teuthology.orchestra.run.smithi090:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-23T01:44:31.456 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 0a731bfd-0fc4-4c1b-8e4b-f6dfbc059bcc... 2024-04-23T01:44:31.457 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-23T01:44:31.499 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-23T01:44:31.536 INFO:teuthology.orchestra.run.smithi090.stdout:[global] 2024-04-23T01:44:31.537 INFO:teuthology.orchestra.run.smithi090.stdout: chdir = "" 2024-04-23T01:44:31.537 INFO:teuthology.orchestra.run.smithi090.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-23T01:44:31.537 INFO:teuthology.orchestra.run.smithi090.stdout: auth supported = cephx 2024-04-23T01:44:31.537 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:31.537 INFO:teuthology.orchestra.run.smithi090.stdout: filestore xattr use omap = true 2024-04-23T01:44:31.537 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:31.537 INFO:teuthology.orchestra.run.smithi090.stdout: mon clock drift allowed = 1.000 2024-04-23T01:44:31.537 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:31.537 INFO:teuthology.orchestra.run.smithi090.stdout: osd crush chooseleaf type = 0 2024-04-23T01:44:31.537 INFO:teuthology.orchestra.run.smithi090.stdout: auth debug = true 2024-04-23T01:44:31.537 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:31.537 INFO:teuthology.orchestra.run.smithi090.stdout: ms die on old message = true 2024-04-23T01:44:31.537 INFO:teuthology.orchestra.run.smithi090.stdout: ms die on bug = true 2024-04-23T01:44:31.537 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:31.537 INFO:teuthology.orchestra.run.smithi090.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-23T01:44:31.538 INFO:teuthology.orchestra.run.smithi090.stdout: mon pg warn max object skew = 0 2024-04-23T01:44:31.538 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:31.538 INFO:teuthology.orchestra.run.smithi090.stdout: # disable pg_autoscaler by default for new pools 2024-04-23T01:44:31.538 INFO:teuthology.orchestra.run.smithi090.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-23T01:44:31.538 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:31.538 INFO:teuthology.orchestra.run.smithi090.stdout: osd pool default size = 2 2024-04-23T01:44:31.538 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:31.538 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd allow primary affinity = true 2024-04-23T01:44:31.538 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd allow pg remap = true 2024-04-23T01:44:31.538 INFO:teuthology.orchestra.run.smithi090.stdout: mon warn on legacy crush tunables = false 2024-04-23T01:44:31.538 INFO:teuthology.orchestra.run.smithi090.stdout: mon warn on crush straw calc version zero = false 2024-04-23T01:44:31.538 INFO:teuthology.orchestra.run.smithi090.stdout: mon warn on no sortbitwise = false 2024-04-23T01:44:31.538 INFO:teuthology.orchestra.run.smithi090.stdout: mon warn on osd down out interval zero = false 2024-04-23T01:44:31.538 INFO:teuthology.orchestra.run.smithi090.stdout: mon warn on too few osds = false 2024-04-23T01:44:31.539 INFO:teuthology.orchestra.run.smithi090.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-23T01:44:31.539 INFO:teuthology.orchestra.run.smithi090.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-23T01:44:31.539 INFO:teuthology.orchestra.run.smithi090.stdout: mon_allow_pool_size_one = true 2024-04-23T01:44:31.539 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:31.539 INFO:teuthology.orchestra.run.smithi090.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-23T01:44:31.539 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:31.539 INFO:teuthology.orchestra.run.smithi090.stdout: osd default data pool replay window = 5 2024-04-23T01:44:31.539 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:31.539 INFO:teuthology.orchestra.run.smithi090.stdout: mon allow pool delete = true 2024-04-23T01:44:31.539 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:31.539 INFO:teuthology.orchestra.run.smithi090.stdout: mon cluster log file level = debug 2024-04-23T01:44:31.539 INFO:teuthology.orchestra.run.smithi090.stdout: debug asserts on shutdown = true 2024-04-23T01:44:31.539 INFO:teuthology.orchestra.run.smithi090.stdout: mon health detail to clog = false 2024-04-23T01:44:31.539 INFO:teuthology.orchestra.run.smithi090.stdout: mon host = 172.21.15.90 2024-04-23T01:44:31.539 INFO:teuthology.orchestra.run.smithi090.stdout: enable experimental unrecoverable data corrupting features = crimson 2024-04-23T01:44:31.540 INFO:teuthology.orchestra.run.smithi090.stdout: ms client mode = crc 2024-04-23T01:44:31.540 INFO:teuthology.orchestra.run.smithi090.stdout: ms cluster mode = crc 2024-04-23T01:44:31.540 INFO:teuthology.orchestra.run.smithi090.stdout: ms mon client mode = crc 2024-04-23T01:44:31.540 INFO:teuthology.orchestra.run.smithi090.stdout: ms mon cluster mode = crc 2024-04-23T01:44:31.540 INFO:teuthology.orchestra.run.smithi090.stdout: ms mon service mode = crc 2024-04-23T01:44:31.540 INFO:teuthology.orchestra.run.smithi090.stdout: ms service mode = crc 2024-04-23T01:44:31.540 INFO:teuthology.orchestra.run.smithi090.stdout: osd_pool_default_min_size = 2 2024-04-23T01:44:31.540 INFO:teuthology.orchestra.run.smithi090.stdout: osd_pool_default_size = 3 2024-04-23T01:44:31.540 INFO:teuthology.orchestra.run.smithi090.stdout: fsid = 0a731bfd-0fc4-4c1b-8e4b-f6dfbc059bcc 2024-04-23T01:44:31.540 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:31.540 INFO:teuthology.orchestra.run.smithi090.stdout:[osd] 2024-04-23T01:44:31.540 INFO:teuthology.orchestra.run.smithi090.stdout: osd journal size = 100 2024-04-23T01:44:31.540 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:31.540 INFO:teuthology.orchestra.run.smithi090.stdout: osd scrub load threshold = 5.0 2024-04-23T01:44:31.540 INFO:teuthology.orchestra.run.smithi090.stdout: osd scrub max interval = 120 2024-04-23T01:44:31.541 INFO:teuthology.orchestra.run.smithi090.stdout: osd mclock profile = high_recovery_ops 2024-04-23T01:44:31.541 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:31.541 INFO:teuthology.orchestra.run.smithi090.stdout: osd recover clone overlap = true 2024-04-23T01:44:31.541 INFO:teuthology.orchestra.run.smithi090.stdout: osd recovery max chunk = 1048576 2024-04-23T01:44:31.541 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:31.541 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug shutdown = true 2024-04-23T01:44:31.541 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug op order = true 2024-04-23T01:44:31.541 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug verify stray on activate = true 2024-04-23T01:44:31.541 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug trim objects = true 2024-04-23T01:44:31.541 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:31.541 INFO:teuthology.orchestra.run.smithi090.stdout: osd open classes on start = true 2024-04-23T01:44:31.541 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug pg log writeout = true 2024-04-23T01:44:31.541 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:31.541 INFO:teuthology.orchestra.run.smithi090.stdout: osd deep scrub update digest min age = 30 2024-04-23T01:44:31.541 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:31.542 INFO:teuthology.orchestra.run.smithi090.stdout: osd map max advance = 10 2024-04-23T01:44:31.542 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:31.542 INFO:teuthology.orchestra.run.smithi090.stdout: journal zero on create = true 2024-04-23T01:44:31.542 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:31.542 INFO:teuthology.orchestra.run.smithi090.stdout: filestore ondisk finisher threads = 3 2024-04-23T01:44:31.542 INFO:teuthology.orchestra.run.smithi090.stdout: filestore apply finisher threads = 3 2024-04-23T01:44:31.542 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:31.542 INFO:teuthology.orchestra.run.smithi090.stdout: bdev debug aio = true 2024-04-23T01:44:31.542 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug misdirected ops = true 2024-04-23T01:44:31.542 INFO:teuthology.orchestra.run.smithi090.stdout: debug monc = 20 2024-04-23T01:44:31.542 INFO:teuthology.orchestra.run.smithi090.stdout: bluestore block size = 96636764160 2024-04-23T01:44:31.542 INFO:teuthology.orchestra.run.smithi090.stdout: bluestore compression algorithm = snappy 2024-04-23T01:44:31.542 INFO:teuthology.orchestra.run.smithi090.stdout: bluestore compression mode = aggressive 2024-04-23T01:44:31.542 INFO:teuthology.orchestra.run.smithi090.stdout: bluestore fsck on mount = True 2024-04-23T01:44:31.542 INFO:teuthology.orchestra.run.smithi090.stdout: bluestore rocksdb cf = False 2024-04-23T01:44:31.543 INFO:teuthology.orchestra.run.smithi090.stdout: crimson alien thread cpu cores = 6-7 2024-04-23T01:44:31.543 INFO:teuthology.orchestra.run.smithi090.stdout: crimson osd obc lru size = 10 2024-04-23T01:44:31.543 INFO:teuthology.orchestra.run.smithi090.stdout: debug bluefs = 20 2024-04-23T01:44:31.543 INFO:teuthology.orchestra.run.smithi090.stdout: debug bluestore = 20 2024-04-23T01:44:31.543 INFO:teuthology.orchestra.run.smithi090.stdout: debug ms = 1 2024-04-23T01:44:31.543 INFO:teuthology.orchestra.run.smithi090.stdout: debug osd = 20 2024-04-23T01:44:31.543 INFO:teuthology.orchestra.run.smithi090.stdout: debug rocksdb = 10 2024-04-23T01:44:31.543 INFO:teuthology.orchestra.run.smithi090.stdout: err to stderr = True 2024-04-23T01:44:31.543 INFO:teuthology.orchestra.run.smithi090.stdout: log flush on exit = True 2024-04-23T01:44:31.543 INFO:teuthology.orchestra.run.smithi090.stdout: log to file = False 2024-04-23T01:44:31.543 INFO:teuthology.orchestra.run.smithi090.stdout: log to stderr = True 2024-04-23T01:44:31.543 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd backfillfull_ratio = 0.85 2024-04-23T01:44:31.543 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd full ratio = 0.9 2024-04-23T01:44:31.543 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd nearfull ratio = 0.8 2024-04-23T01:44:31.543 INFO:teuthology.orchestra.run.smithi090.stdout: osd blocked scrub grace period = 3600 2024-04-23T01:44:31.543 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug reject backfill probability = 0.3 2024-04-23T01:44:31.543 INFO:teuthology.orchestra.run.smithi090.stdout: osd delete sleep = 1 2024-04-23T01:44:31.544 INFO:teuthology.orchestra.run.smithi090.stdout: osd failsafe full ratio = 0.95 2024-04-23T01:44:31.544 INFO:teuthology.orchestra.run.smithi090.stdout: osd max backfills = 3 2024-04-23T01:44:31.544 INFO:teuthology.orchestra.run.smithi090.stdout: osd max markdown count = 1000 2024-04-23T01:44:31.544 INFO:teuthology.orchestra.run.smithi090.stdout: osd objectstore = bluestore 2024-04-23T01:44:31.544 INFO:teuthology.orchestra.run.smithi090.stdout: osd scrub min interval = 60 2024-04-23T01:44:31.544 INFO:teuthology.orchestra.run.smithi090.stdout: osd shutdown pgref assert = True 2024-04-23T01:44:31.544 INFO:teuthology.orchestra.run.smithi090.stdout: osd snap trim sleep = 2 2024-04-23T01:44:31.544 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:31.544 INFO:teuthology.orchestra.run.smithi090.stdout:[mgr] 2024-04-23T01:44:31.544 INFO:teuthology.orchestra.run.smithi090.stdout: debug ms = 1 2024-04-23T01:44:31.544 INFO:teuthology.orchestra.run.smithi090.stdout: debug mgr = 20 2024-04-23T01:44:31.544 INFO:teuthology.orchestra.run.smithi090.stdout: debug mon = 20 2024-04-23T01:44:31.544 INFO:teuthology.orchestra.run.smithi090.stdout: debug auth = 20 2024-04-23T01:44:31.544 INFO:teuthology.orchestra.run.smithi090.stdout: mon reweight min pgs per osd = 4 2024-04-23T01:44:31.544 INFO:teuthology.orchestra.run.smithi090.stdout: mon reweight min bytes per osd = 10 2024-04-23T01:44:31.545 INFO:teuthology.orchestra.run.smithi090.stdout: mgr/telemetry/nag = false 2024-04-23T01:44:31.545 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:31.545 INFO:teuthology.orchestra.run.smithi090.stdout:[mon] 2024-04-23T01:44:31.545 INFO:teuthology.orchestra.run.smithi090.stdout: debug ms = 1 2024-04-23T01:44:31.545 INFO:teuthology.orchestra.run.smithi090.stdout: debug mon = 20 2024-04-23T01:44:31.545 INFO:teuthology.orchestra.run.smithi090.stdout: debug paxos = 20 2024-04-23T01:44:31.545 INFO:teuthology.orchestra.run.smithi090.stdout: debug auth = 20 2024-04-23T01:44:31.545 INFO:teuthology.orchestra.run.smithi090.stdout: mon data avail warn = 5 2024-04-23T01:44:31.545 INFO:teuthology.orchestra.run.smithi090.stdout: mon mgr mkfs grace = 240 2024-04-23T01:44:31.545 INFO:teuthology.orchestra.run.smithi090.stdout: mon reweight min pgs per osd = 4 2024-04-23T01:44:31.545 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd reporter subtree level = osd 2024-04-23T01:44:31.545 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd prime pg temp = true 2024-04-23T01:44:31.545 INFO:teuthology.orchestra.run.smithi090.stdout: mon reweight min bytes per osd = 10 2024-04-23T01:44:31.545 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:31.545 INFO:teuthology.orchestra.run.smithi090.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-23T01:44:31.546 INFO:teuthology.orchestra.run.smithi090.stdout: auth mon ticket ttl = 660 # 11m 2024-04-23T01:44:31.546 INFO:teuthology.orchestra.run.smithi090.stdout: auth service ticket ttl = 240 # 4m 2024-04-23T01:44:31.546 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:31.546 INFO:teuthology.orchestra.run.smithi090.stdout: # don't complain about insecure global_id in the test suite 2024-04-23T01:44:31.546 INFO:teuthology.orchestra.run.smithi090.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-23T01:44:31.546 INFO:teuthology.orchestra.run.smithi090.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-23T01:44:31.546 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:31.546 INFO:teuthology.orchestra.run.smithi090.stdout: # 1m isn't quite enough 2024-04-23T01:44:31.546 INFO:teuthology.orchestra.run.smithi090.stdout: mon_down_mkfs_grace = 2m 2024-04-23T01:44:31.546 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:31.546 INFO:teuthology.orchestra.run.smithi090.stdout: mon_warn_on_filestore_osds = false 2024-04-23T01:44:31.546 INFO:teuthology.orchestra.run.smithi090.stdout: mon min osdmap epochs = 50 2024-04-23T01:44:31.546 INFO:teuthology.orchestra.run.smithi090.stdout: mon osdmap full prune interval = 2 2024-04-23T01:44:31.546 INFO:teuthology.orchestra.run.smithi090.stdout: mon osdmap full prune min = 15 2024-04-23T01:44:31.547 INFO:teuthology.orchestra.run.smithi090.stdout: mon osdmap full prune txsize = 2 2024-04-23T01:44:31.547 INFO:teuthology.orchestra.run.smithi090.stdout: osd pool default crimson = True 2024-04-23T01:44:31.547 INFO:teuthology.orchestra.run.smithi090.stdout: paxos service trim min = 10 2024-04-23T01:44:31.547 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:44:31.547 INFO:teuthology.orchestra.run.smithi090.stdout:[client] 2024-04-23T01:44:31.547 INFO:teuthology.orchestra.run.smithi090.stdout: rgw cache enabled = true 2024-04-23T01:44:31.547 INFO:teuthology.orchestra.run.smithi090.stdout: rgw enable ops log = true 2024-04-23T01:44:31.547 INFO:teuthology.orchestra.run.smithi090.stdout: rgw enable usage log = true 2024-04-23T01:44:31.547 INFO:teuthology.orchestra.run.smithi090.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-23T01:44:31.547 INFO:teuthology.orchestra.run.smithi090.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-23T01:44:31.547 INFO:teuthology.orchestra.run.smithi090.stdout:[mon.a] 2024-04-23T01:44:31.547 INFO:teuthology.orchestra.run.smithi090.stdout:[client.0] 2024-04-23T01:44:31.547 INFO:teuthology.orchestra.run.smithi090.stdout: debug ms = 1 2024-04-23T01:44:31.547 INFO:teuthology.orchestra.run.smithi090.stdout: debug objecter = 20 2024-04-23T01:44:31.547 INFO:teuthology.orchestra.run.smithi090.stdout: debug rados = 20 2024-04-23T01:44:31.547 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.0] 2024-04-23T01:44:31.548 INFO:teuthology.orchestra.run.smithi090.stdout: crimson seastar cpu cores = 0-2 2024-04-23T01:44:31.548 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.1] 2024-04-23T01:44:31.548 INFO:teuthology.orchestra.run.smithi090.stdout: crimson seastar cpu cores = 3-5 2024-04-23T01:44:31.548 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.2] 2024-04-23T01:44:31.548 INFO:teuthology.orchestra.run.smithi090.stdout: crimson seastar cpu cores = 0-2 2024-04-23T01:44:31.548 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.3] 2024-04-23T01:44:31.548 INFO:teuthology.orchestra.run.smithi090.stdout: crimson seastar cpu cores = 3-5 2024-04-23T01:44:31.548 INFO:teuthology.orchestra.run.smithi113.stdout:[global] 2024-04-23T01:44:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: chdir = "" 2024-04-23T01:44:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-23T01:44:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: auth supported = cephx 2024-04-23T01:44:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: filestore xattr use omap = true 2024-04-23T01:44:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: mon clock drift allowed = 1.000 2024-04-23T01:44:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: osd crush chooseleaf type = 0 2024-04-23T01:44:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: auth debug = true 2024-04-23T01:44:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on old message = true 2024-04-23T01:44:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on bug = true 2024-04-23T01:44:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-23T01:44:31.550 INFO:teuthology.orchestra.run.smithi113.stdout: mon pg warn max object skew = 0 2024-04-23T01:44:31.550 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:31.550 INFO:teuthology.orchestra.run.smithi113.stdout: # disable pg_autoscaler by default for new pools 2024-04-23T01:44:31.550 INFO:teuthology.orchestra.run.smithi113.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-23T01:44:31.550 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:31.550 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default size = 2 2024-04-23T01:44:31.550 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:31.550 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd allow primary affinity = true 2024-04-23T01:44:31.550 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd allow pg remap = true 2024-04-23T01:44:31.550 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on legacy crush tunables = false 2024-04-23T01:44:31.550 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on crush straw calc version zero = false 2024-04-23T01:44:31.550 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on no sortbitwise = false 2024-04-23T01:44:31.550 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on osd down out interval zero = false 2024-04-23T01:44:31.550 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on too few osds = false 2024-04-23T01:44:31.550 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-23T01:44:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-23T01:44:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: mon_allow_pool_size_one = true 2024-04-23T01:44:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-23T01:44:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: osd default data pool replay window = 5 2024-04-23T01:44:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: mon allow pool delete = true 2024-04-23T01:44:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: mon cluster log file level = debug 2024-04-23T01:44:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: debug asserts on shutdown = true 2024-04-23T01:44:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: mon health detail to clog = false 2024-04-23T01:44:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: mon host = 172.21.15.90 2024-04-23T01:44:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: enable experimental unrecoverable data corrupting features = crimson 2024-04-23T01:44:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: ms client mode = crc 2024-04-23T01:44:31.552 INFO:teuthology.orchestra.run.smithi113.stdout: ms cluster mode = crc 2024-04-23T01:44:31.552 INFO:teuthology.orchestra.run.smithi113.stdout: ms mon client mode = crc 2024-04-23T01:44:31.552 INFO:teuthology.orchestra.run.smithi113.stdout: ms mon cluster mode = crc 2024-04-23T01:44:31.552 INFO:teuthology.orchestra.run.smithi113.stdout: ms mon service mode = crc 2024-04-23T01:44:31.552 INFO:teuthology.orchestra.run.smithi113.stdout: ms service mode = crc 2024-04-23T01:44:31.552 INFO:teuthology.orchestra.run.smithi113.stdout: osd_pool_default_min_size = 2 2024-04-23T01:44:31.552 INFO:teuthology.orchestra.run.smithi113.stdout: osd_pool_default_size = 3 2024-04-23T01:44:31.552 INFO:teuthology.orchestra.run.smithi113.stdout: fsid = 0a731bfd-0fc4-4c1b-8e4b-f6dfbc059bcc 2024-04-23T01:44:31.552 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:31.552 INFO:teuthology.orchestra.run.smithi113.stdout:[osd] 2024-04-23T01:44:31.552 INFO:teuthology.orchestra.run.smithi113.stdout: osd journal size = 100 2024-04-23T01:44:31.552 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:31.552 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub load threshold = 5.0 2024-04-23T01:44:31.552 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub max interval = 120 2024-04-23T01:44:31.552 INFO:teuthology.orchestra.run.smithi113.stdout: osd mclock profile = high_recovery_ops 2024-04-23T01:44:31.552 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:31.552 INFO:teuthology.orchestra.run.smithi113.stdout: osd recover clone overlap = true 2024-04-23T01:44:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: osd recovery max chunk = 1048576 2024-04-23T01:44:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug shutdown = true 2024-04-23T01:44:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug op order = true 2024-04-23T01:44:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug verify stray on activate = true 2024-04-23T01:44:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug trim objects = true 2024-04-23T01:44:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: osd open classes on start = true 2024-04-23T01:44:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug pg log writeout = true 2024-04-23T01:44:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: osd deep scrub update digest min age = 30 2024-04-23T01:44:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: osd map max advance = 10 2024-04-23T01:44:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: journal zero on create = true 2024-04-23T01:44:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:31.554 INFO:teuthology.orchestra.run.smithi113.stdout: filestore ondisk finisher threads = 3 2024-04-23T01:44:31.554 INFO:teuthology.orchestra.run.smithi113.stdout: filestore apply finisher threads = 3 2024-04-23T01:44:31.554 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:31.554 INFO:teuthology.orchestra.run.smithi113.stdout: bdev debug aio = true 2024-04-23T01:44:31.554 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug misdirected ops = true 2024-04-23T01:44:31.554 INFO:teuthology.orchestra.run.smithi113.stdout: debug monc = 20 2024-04-23T01:44:31.554 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore block size = 96636764160 2024-04-23T01:44:31.554 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore compression algorithm = snappy 2024-04-23T01:44:31.554 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore compression mode = aggressive 2024-04-23T01:44:31.554 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore fsck on mount = True 2024-04-23T01:44:31.554 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore rocksdb cf = False 2024-04-23T01:44:31.554 INFO:teuthology.orchestra.run.smithi113.stdout: crimson alien thread cpu cores = 6-7 2024-04-23T01:44:31.554 INFO:teuthology.orchestra.run.smithi113.stdout: crimson osd obc lru size = 10 2024-04-23T01:44:31.554 INFO:teuthology.orchestra.run.smithi113.stdout: debug bluefs = 20 2024-04-23T01:44:31.554 INFO:teuthology.orchestra.run.smithi113.stdout: debug bluestore = 20 2024-04-23T01:44:31.555 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-23T01:44:31.555 INFO:teuthology.orchestra.run.smithi113.stdout: debug osd = 20 2024-04-23T01:44:31.555 INFO:teuthology.orchestra.run.smithi113.stdout: debug rocksdb = 10 2024-04-23T01:44:31.555 INFO:teuthology.orchestra.run.smithi113.stdout: err to stderr = True 2024-04-23T01:44:31.555 INFO:teuthology.orchestra.run.smithi113.stdout: log flush on exit = True 2024-04-23T01:44:31.555 INFO:teuthology.orchestra.run.smithi113.stdout: log to file = False 2024-04-23T01:44:31.555 INFO:teuthology.orchestra.run.smithi113.stdout: log to stderr = True 2024-04-23T01:44:31.555 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd backfillfull_ratio = 0.85 2024-04-23T01:44:31.555 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd full ratio = 0.9 2024-04-23T01:44:31.555 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd nearfull ratio = 0.8 2024-04-23T01:44:31.555 INFO:teuthology.orchestra.run.smithi113.stdout: osd blocked scrub grace period = 3600 2024-04-23T01:44:31.555 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug reject backfill probability = 0.3 2024-04-23T01:44:31.555 INFO:teuthology.orchestra.run.smithi113.stdout: osd delete sleep = 1 2024-04-23T01:44:31.555 INFO:teuthology.orchestra.run.smithi113.stdout: osd failsafe full ratio = 0.95 2024-04-23T01:44:31.556 INFO:teuthology.orchestra.run.smithi113.stdout: osd max backfills = 3 2024-04-23T01:44:31.556 INFO:teuthology.orchestra.run.smithi113.stdout: osd max markdown count = 1000 2024-04-23T01:44:31.556 INFO:teuthology.orchestra.run.smithi113.stdout: osd objectstore = bluestore 2024-04-23T01:44:31.556 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub min interval = 60 2024-04-23T01:44:31.556 INFO:teuthology.orchestra.run.smithi113.stdout: osd shutdown pgref assert = True 2024-04-23T01:44:31.556 INFO:teuthology.orchestra.run.smithi113.stdout: osd snap trim sleep = 2 2024-04-23T01:44:31.556 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:31.556 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr] 2024-04-23T01:44:31.556 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-23T01:44:31.556 INFO:teuthology.orchestra.run.smithi113.stdout: debug mgr = 20 2024-04-23T01:44:31.556 INFO:teuthology.orchestra.run.smithi113.stdout: debug mon = 20 2024-04-23T01:44:31.556 INFO:teuthology.orchestra.run.smithi113.stdout: debug auth = 20 2024-04-23T01:44:31.556 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min pgs per osd = 4 2024-04-23T01:44:31.556 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min bytes per osd = 10 2024-04-23T01:44:31.556 INFO:teuthology.orchestra.run.smithi113.stdout: mgr/telemetry/nag = false 2024-04-23T01:44:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:31.557 INFO:teuthology.orchestra.run.smithi113.stdout:[mon] 2024-04-23T01:44:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-23T01:44:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: debug mon = 20 2024-04-23T01:44:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: debug paxos = 20 2024-04-23T01:44:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: debug auth = 20 2024-04-23T01:44:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: mon data avail warn = 5 2024-04-23T01:44:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: mon mgr mkfs grace = 240 2024-04-23T01:44:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min pgs per osd = 4 2024-04-23T01:44:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd reporter subtree level = osd 2024-04-23T01:44:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd prime pg temp = true 2024-04-23T01:44:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min bytes per osd = 10 2024-04-23T01:44:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-23T01:44:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: auth mon ticket ttl = 660 # 11m 2024-04-23T01:44:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: auth service ticket ttl = 240 # 4m 2024-04-23T01:44:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:31.558 INFO:teuthology.orchestra.run.smithi113.stdout: # don't complain about insecure global_id in the test suite 2024-04-23T01:44:31.558 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-23T01:44:31.558 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-23T01:44:31.558 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:31.558 INFO:teuthology.orchestra.run.smithi113.stdout: # 1m isn't quite enough 2024-04-23T01:44:31.558 INFO:teuthology.orchestra.run.smithi113.stdout: mon_down_mkfs_grace = 2m 2024-04-23T01:44:31.558 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:31.558 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_filestore_osds = false 2024-04-23T01:44:31.558 INFO:teuthology.orchestra.run.smithi113.stdout: mon min osdmap epochs = 50 2024-04-23T01:44:31.558 INFO:teuthology.orchestra.run.smithi113.stdout: mon osdmap full prune interval = 2 2024-04-23T01:44:31.558 INFO:teuthology.orchestra.run.smithi113.stdout: mon osdmap full prune min = 15 2024-04-23T01:44:31.558 INFO:teuthology.orchestra.run.smithi113.stdout: mon osdmap full prune txsize = 2 2024-04-23T01:44:31.558 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default crimson = True 2024-04-23T01:44:31.558 INFO:teuthology.orchestra.run.smithi113.stdout: paxos service trim min = 10 2024-04-23T01:44:31.558 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-23T01:44:31.559 INFO:teuthology.orchestra.run.smithi113.stdout:[client] 2024-04-23T01:44:31.559 INFO:teuthology.orchestra.run.smithi113.stdout: rgw cache enabled = true 2024-04-23T01:44:31.559 INFO:teuthology.orchestra.run.smithi113.stdout: rgw enable ops log = true 2024-04-23T01:44:31.559 INFO:teuthology.orchestra.run.smithi113.stdout: rgw enable usage log = true 2024-04-23T01:44:31.559 INFO:teuthology.orchestra.run.smithi113.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-23T01:44:31.559 INFO:teuthology.orchestra.run.smithi113.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-23T01:44:31.559 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.a] 2024-04-23T01:44:31.559 INFO:teuthology.orchestra.run.smithi113.stdout:[client.0] 2024-04-23T01:44:31.559 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-23T01:44:31.559 INFO:teuthology.orchestra.run.smithi113.stdout: debug objecter = 20 2024-04-23T01:44:31.559 INFO:teuthology.orchestra.run.smithi113.stdout: debug rados = 20 2024-04-23T01:44:31.559 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.0] 2024-04-23T01:44:31.559 INFO:teuthology.orchestra.run.smithi113.stdout: crimson seastar cpu cores = 0-2 2024-04-23T01:44:31.559 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.1] 2024-04-23T01:44:31.559 INFO:teuthology.orchestra.run.smithi113.stdout: crimson seastar cpu cores = 3-5 2024-04-23T01:44:31.559 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.2] 2024-04-23T01:44:31.560 INFO:teuthology.orchestra.run.smithi113.stdout: crimson seastar cpu cores = 0-2 2024-04-23T01:44:31.560 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.3] 2024-04-23T01:44:31.560 INFO:teuthology.orchestra.run.smithi113.stdout: crimson seastar cpu cores = 3-5 2024-04-23T01:44:31.561 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-23T01:44:31.561 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-23T01:44:31.663 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-23T01:44:31.663 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-23T01:44:31.663 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-23T01:44:31.678 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-23T01:44:31.678 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-23T01:44:31.732 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi090.front.sepia.ceph.com 2024-04-23T01:44:31.732 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-23T01:44:31.732 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-23T01:44:31.732 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-23T01:44:31.804 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-23T01:44:31.804 DEBUG:teuthology.orchestra.run.smithi090:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-23T01:44:31.858 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi113.front.sepia.ceph.com 2024-04-23T01:44:31.859 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-23T01:44:31.859 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-23T01:44:31.859 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-23T01:44:31.893 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-23T01:44:31.893 DEBUG:teuthology.orchestra.run.smithi113:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-23T01:44:31.948 INFO:tasks.ceph:Setting up mon nodes... 2024-04-23T01:44:31.948 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-23T01:44:31.949 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-23T01:44:32.081 INFO:teuthology.orchestra.run.smithi113.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-23T01:44:32.084 INFO:tasks.ceph:Setting up mds nodes... 2024-04-23T01:44:32.084 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-23T01:44:32.085 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-23T01:44:32.133 INFO:teuthology.orchestra.run.smithi090.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-23T01:44:32.145 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-23T01:44:32.192 INFO:teuthology.orchestra.run.smithi113.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-23T01:44:32.204 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-23T01:44:32.204 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi090.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi113.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2024-04-23T01:44:32.205 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-23T01:44:32.228 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-04-23T01:44:32.228 INFO:tasks.ceph:role: osd.0 2024-04-23T01:44:32.228 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi090.front.sepia.ceph.com 2024-04-23T01:44:32.228 DEBUG:teuthology.orchestra.run.smithi090:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-23T01:44:32.298 INFO:teuthology.orchestra.run.smithi090.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T01:44:32.298 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T01:44:32.299 INFO:teuthology.orchestra.run.smithi090.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T01:44:32.299 INFO:teuthology.orchestra.run.smithi090.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-23T01:44:32.299 INFO:teuthology.orchestra.run.smithi090.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T01:44:32.299 INFO:teuthology.orchestra.run.smithi090.stdout: = sunit=0 swidth=0 blks 2024-04-23T01:44:32.299 INFO:teuthology.orchestra.run.smithi090.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T01:44:32.299 INFO:teuthology.orchestra.run.smithi090.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-23T01:44:32.299 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T01:44:32.299 INFO:teuthology.orchestra.run.smithi090.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T01:44:32.809 INFO:teuthology.orchestra.run.smithi090.stdout:Discarding blocks...Done. 2024-04-23T01:44:32.811 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi090.front.sepia.ceph.com -o noatime 2024-04-23T01:44:32.811 DEBUG:teuthology.orchestra.run.smithi090:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-23T01:44:32.936 DEBUG:teuthology.orchestra.run.smithi090:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-23T01:44:32.967 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-23T01:44:33.032 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-04-23T01:44:33.032 INFO:tasks.ceph:role: osd.1 2024-04-23T01:44:33.032 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi090.front.sepia.ceph.com 2024-04-23T01:44:33.032 DEBUG:teuthology.orchestra.run.smithi090:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-23T01:44:33.103 INFO:teuthology.orchestra.run.smithi090.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T01:44:33.104 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T01:44:33.104 INFO:teuthology.orchestra.run.smithi090.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T01:44:33.104 INFO:teuthology.orchestra.run.smithi090.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-23T01:44:33.104 INFO:teuthology.orchestra.run.smithi090.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T01:44:33.104 INFO:teuthology.orchestra.run.smithi090.stdout: = sunit=0 swidth=0 blks 2024-04-23T01:44:33.104 INFO:teuthology.orchestra.run.smithi090.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T01:44:33.104 INFO:teuthology.orchestra.run.smithi090.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-23T01:44:33.104 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T01:44:33.104 INFO:teuthology.orchestra.run.smithi090.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T01:44:33.568 INFO:teuthology.orchestra.run.smithi090.stdout:Discarding blocks...Done. 2024-04-23T01:44:33.570 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi090.front.sepia.ceph.com -o noatime 2024-04-23T01:44:33.570 DEBUG:teuthology.orchestra.run.smithi090:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-23T01:44:33.653 DEBUG:teuthology.orchestra.run.smithi090:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-23T01:44:33.722 DEBUG:teuthology.orchestra.run.smithi090:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T01:44:34.009 INFO:teuthology.orchestra.run.smithi090.stderr:WARNING: debug mode. Not for benchmarking or production 2024-04-23T01:44:34.010 INFO:teuthology.orchestra.run.smithi090.stderr:INFO 2024-04-23 01:44:34,013 seastar - Reactor backend: linux-aio 2024-04-23T01:44:34.026 INFO:teuthology.orchestra.run.smithi090.stderr:==26224==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-04-23T01:44:34.128 INFO:teuthology.orchestra.run.smithi090.stderr:Reactor stalled for 65 ms on shard 0. Backtrace: 0x6bddd 0xb9d9299 0xb8abd60 0xb8ad2dc 0xb8ad4f2 0xb8ad648 0xb8adb11 0x3e6ef 0x4a0b2 0x35b7c 0x3629e 0x348a7b9 0x5dd0dcf 0x5dd11d4 0x5dd15b6 0x5e2d544 0x5dac0c0 0x5dac5f4 0x5bf135d 0x5c0c421 0x6ee4d94 0x6ef6567 0x6ef7a04 0x6ef7a89 0x6ef7b8b 0x6ef7c83 0x6ef7e90 0x6ef8076 0x6ef83c5 0x6ef8769 0x6ef8999 0x6ef8c03 0x6ef8ed0 0x6ef90ee 0x6ef91a6 0x6ef926c 0x6ef934d 0x5a59c56 0x5a59d60 0x5a59e1c 0x5a59ed8 0x5a59f94 0x5a5a0e1 0x5a5a26c 0x5a5a62d 0x5a7656e 0xb8be9e5 0xb8d8c7f 0xb979c7d 0xb97b2cb 0xb657ab1 0xb65842d 0x37d891a 0x37d9473 0x3678d7a 0x2958f 0x2963f 0x3465444 2024-04-23T01:44:34.128 INFO:teuthology.orchestra.run.smithi090.stderr:kernel callstack: 2024-04-23T01:44:34.137 INFO:teuthology.orchestra.run.smithi090.stderr:INFO 2024-04-23 01:44:34,140 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-04-23T01:44:34.137 INFO:teuthology.orchestra.run.smithi090.stderr:INFO 2024-04-23 01:44:34,140 [shard 0:main] osd - get_early_config: set --cpuset 0-2 2024-04-23T01:44:34.138 INFO:teuthology.orchestra.run.smithi090.stderr:DEBUG 2024-04-23 01:44:34,141 [shard 0:main] seastar - reactor::drain 2024-04-23T01:44:34.686 INFO:teuthology.orchestra.run.smithi090.stderr:----------------------------------------------------- 2024-04-23T01:44:34.686 INFO:teuthology.orchestra.run.smithi090.stderr:Suppressions used: 2024-04-23T01:44:34.686 INFO:teuthology.orchestra.run.smithi090.stderr: count bytes template 2024-04-23T01:44:34.686 INFO:teuthology.orchestra.run.smithi090.stderr: 1 8 MallocExtension::Register 2024-04-23T01:44:34.687 INFO:teuthology.orchestra.run.smithi090.stderr:----------------------------------------------------- 2024-04-23T01:44:34.687 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-04-23T01:44:34.745 INFO:teuthology.orchestra.run.smithi090.stderr:WARNING: debug mode. Not for benchmarking or production 2024-04-23T01:44:34.749 INFO:teuthology.orchestra.run.smithi090.stderr:INFO 2024-04-23 01:44:34,751 seastar - Reactor backend: linux-aio 2024-04-23T01:44:34.757 INFO:teuthology.orchestra.run.smithi090.stderr:==26222==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-04-23T01:44:34.822 INFO:teuthology.orchestra.run.smithi090.stderr:Reactor stalled for 65 ms on shard 0. Backtrace: 0x6bddd 0xb9d9299 0xb8abd60 0xb8ad2dc 0xb8ad4f2 0xb8ad648 0xb8adb11 0x3e6ef 0x113c67 0x4a0bd 0x35b7c 0x3629e 0x5dde4c4 0x5dde68c 0x5dde8bd 0x5ddeb93 0x5e2d3d3 0x5e2d5d9 0x5dac0c0 0x5dac5f4 0x5bf135d 0x5c0c421 0x6ee4d94 0x6ef6567 0x6ef7a04 0x6ef7a89 0x6ef7b8b 0x6ef7c83 0x6ef7e90 0x6ef8076 0x6ef83c5 0x6ef8769 0x6ef8999 0x6ef8c03 0x6ef8ed0 0x6ef90ee 0x6ef91a6 0x6ef926c 0x6ef934d 0x5a59c56 0x5a59d60 0x5a59e1c 0x5a59ed8 0x5a59f94 0x5a5a0e1 0x5a5a26c 0x5a5a62d 0x5a7656e 0xb8be9e5 0xb8d8c7f 0xb979c7d 0xb97b2cb 0xb657ab1 0xb65842d 0x367958a 0x2958f 0x2963f 0x3465444 2024-04-23T01:44:34.822 INFO:teuthology.orchestra.run.smithi090.stderr:kernel callstack: 2024-04-23T01:44:39.401 INFO:teuthology.orchestra.run.smithi090.stderr:----------------------------------------------------- 2024-04-23T01:44:39.401 INFO:teuthology.orchestra.run.smithi090.stderr:Suppressions used: 2024-04-23T01:44:39.401 INFO:teuthology.orchestra.run.smithi090.stderr: count bytes template 2024-04-23T01:44:39.401 INFO:teuthology.orchestra.run.smithi090.stderr: 1 8 MallocExtension::Register 2024-04-23T01:44:39.401 INFO:teuthology.orchestra.run.smithi090.stderr:----------------------------------------------------- 2024-04-23T01:44:39.401 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-04-23T01:44:39.402 INFO:teuthology.orchestra.run.smithi090.stdout:created object store /var/lib/ceph/osd/ceph-0 for osd.0 fsid 0a731bfd-0fc4-4c1b-8e4b-f6dfbc059bcc 2024-04-23T01:44:39.424 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-23T01:44:39.447 DEBUG:teuthology.orchestra.run.smithi090:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T01:44:39.725 INFO:teuthology.orchestra.run.smithi090.stderr:WARNING: debug mode. Not for benchmarking or production 2024-04-23T01:44:39.726 INFO:teuthology.orchestra.run.smithi090.stderr:INFO 2024-04-23 01:44:39,728 seastar - Reactor backend: linux-aio 2024-04-23T01:44:39.731 INFO:teuthology.orchestra.run.smithi090.stderr:==26406==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-04-23T01:44:39.844 INFO:teuthology.orchestra.run.smithi090.stderr:INFO 2024-04-23 01:44:39,847 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-04-23T01:44:39.845 INFO:teuthology.orchestra.run.smithi090.stderr:INFO 2024-04-23 01:44:39,847 [shard 0:main] osd - get_early_config: set --cpuset 3-5 2024-04-23T01:44:39.845 INFO:teuthology.orchestra.run.smithi090.stderr:DEBUG 2024-04-23 01:44:39,848 [shard 0:main] seastar - reactor::drain 2024-04-23T01:44:40.376 INFO:teuthology.orchestra.run.smithi090.stderr:----------------------------------------------------- 2024-04-23T01:44:40.377 INFO:teuthology.orchestra.run.smithi090.stderr:Suppressions used: 2024-04-23T01:44:40.377 INFO:teuthology.orchestra.run.smithi090.stderr: count bytes template 2024-04-23T01:44:40.377 INFO:teuthology.orchestra.run.smithi090.stderr: 1 8 MallocExtension::Register 2024-04-23T01:44:40.377 INFO:teuthology.orchestra.run.smithi090.stderr:----------------------------------------------------- 2024-04-23T01:44:40.377 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-04-23T01:44:40.438 INFO:teuthology.orchestra.run.smithi090.stderr:WARNING: debug mode. Not for benchmarking or production 2024-04-23T01:44:40.441 INFO:teuthology.orchestra.run.smithi090.stderr:INFO 2024-04-23 01:44:40,444 seastar - Reactor backend: linux-aio 2024-04-23T01:44:40.455 INFO:teuthology.orchestra.run.smithi090.stderr:==26404==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-04-23T01:44:40.564 INFO:teuthology.orchestra.run.smithi090.stderr:Reactor stalled for 65 ms on shard 0. Backtrace: 0x6bddd 0xb9d9299 0xb8abd60 0xb8ad2dc 0xb8ad4f2 0xb8ad648 0xb8adb11 0x3e6ef 0x362f6 0x35e7327 0x5c07d20 0x5c082a8 0x6ee4e03 0x6ef6567 0x6ef7a04 0x6ef7a89 0x6ef7b8b 0x6ef7c83 0x6ef7e90 0x6ef8076 0x6ef83c5 0x6ef8769 0x6ef8999 0x6ef8c03 0x6ef8ed0 0x6ef90ee 0x6ef91a6 0x6ef926c 0x6ef934d 0x5a59c56 0x5a59d60 0x5a59e1c 0x5a59ed8 0x5a59f94 0x5a5a0e1 0x5a5a26c 0x5a5a62d 0x5a7656e 0xb8be9e5 0xb8d8c7f 0xb979c7d 0xb97b2cb 0xb657ab1 0xb65842d 0x367958a 0x2958f 0x2963f 0x3465444 2024-04-23T01:44:40.565 INFO:teuthology.orchestra.run.smithi090.stderr:kernel callstack: 2024-04-23T01:44:45.120 INFO:teuthology.orchestra.run.smithi090.stderr:----------------------------------------------------- 2024-04-23T01:44:45.120 INFO:teuthology.orchestra.run.smithi090.stderr:Suppressions used: 2024-04-23T01:44:45.120 INFO:teuthology.orchestra.run.smithi090.stderr: count bytes template 2024-04-23T01:44:45.120 INFO:teuthology.orchestra.run.smithi090.stderr: 1 8 MallocExtension::Register 2024-04-23T01:44:45.120 INFO:teuthology.orchestra.run.smithi090.stderr:----------------------------------------------------- 2024-04-23T01:44:45.120 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-04-23T01:44:45.121 INFO:teuthology.orchestra.run.smithi090.stdout:created object store /var/lib/ceph/osd/ceph-1 for osd.1 fsid 0a731bfd-0fc4-4c1b-8e4b-f6dfbc059bcc 2024-04-23T01:44:45.142 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-23T01:44:45.165 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-23T01:44:45.193 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-04-23T01:44:45.193 INFO:tasks.ceph:role: osd.2 2024-04-23T01:44:45.193 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi113.front.sepia.ceph.com 2024-04-23T01:44:45.193 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-23T01:44:45.268 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T01:44:45.268 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T01:44:45.268 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T01:44:45.268 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-23T01:44:45.269 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T01:44:45.269 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2024-04-23T01:44:45.269 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T01:44:45.269 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-23T01:44:45.269 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T01:44:45.269 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T01:44:45.777 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2024-04-23T01:44:45.779 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2024-04-23T01:44:45.780 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2024-04-23T01:44:45.878 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-23T01:44:45.911 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-23T01:44:45.976 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-04-23T01:44:45.976 INFO:tasks.ceph:role: osd.3 2024-04-23T01:44:45.976 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi113.front.sepia.ceph.com 2024-04-23T01:44:45.976 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-23T01:44:46.047 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T01:44:46.047 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T01:44:46.047 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T01:44:46.047 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-23T01:44:46.047 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T01:44:46.048 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2024-04-23T01:44:46.048 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T01:44:46.048 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-23T01:44:46.048 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T01:44:46.048 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T01:44:46.482 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2024-04-23T01:44:46.485 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2024-04-23T01:44:46.485 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-3 2024-04-23T01:44:46.552 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-23T01:44:46.621 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T01:44:46.910 INFO:teuthology.orchestra.run.smithi113.stderr:WARNING: debug mode. Not for benchmarking or production 2024-04-23T01:44:46.911 INFO:teuthology.orchestra.run.smithi113.stderr:INFO 2024-04-23 01:44:46,914 seastar - Reactor backend: linux-aio 2024-04-23T01:44:46.926 INFO:teuthology.orchestra.run.smithi113.stderr:==26324==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-04-23T01:44:47.040 INFO:teuthology.orchestra.run.smithi113.stderr:INFO 2024-04-23 01:44:47,042 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-04-23T01:44:47.040 INFO:teuthology.orchestra.run.smithi113.stderr:INFO 2024-04-23 01:44:47,042 [shard 0:main] osd - get_early_config: set --cpuset 0-2 2024-04-23T01:44:47.040 INFO:teuthology.orchestra.run.smithi113.stderr:DEBUG 2024-04-23 01:44:47,043 [shard 0:main] seastar - reactor::drain 2024-04-23T01:44:47.569 INFO:teuthology.orchestra.run.smithi113.stderr:----------------------------------------------------- 2024-04-23T01:44:47.569 INFO:teuthology.orchestra.run.smithi113.stderr:Suppressions used: 2024-04-23T01:44:47.569 INFO:teuthology.orchestra.run.smithi113.stderr: count bytes template 2024-04-23T01:44:47.569 INFO:teuthology.orchestra.run.smithi113.stderr: 1 8 MallocExtension::Register 2024-04-23T01:44:47.569 INFO:teuthology.orchestra.run.smithi113.stderr:----------------------------------------------------- 2024-04-23T01:44:47.569 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-04-23T01:44:47.627 INFO:teuthology.orchestra.run.smithi113.stderr:WARNING: debug mode. Not for benchmarking or production 2024-04-23T01:44:47.631 INFO:teuthology.orchestra.run.smithi113.stderr:INFO 2024-04-23 01:44:47,633 seastar - Reactor backend: linux-aio 2024-04-23T01:44:47.641 INFO:teuthology.orchestra.run.smithi113.stderr:==26322==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-04-23T01:44:52.265 INFO:teuthology.orchestra.run.smithi113.stderr:----------------------------------------------------- 2024-04-23T01:44:52.265 INFO:teuthology.orchestra.run.smithi113.stderr:Suppressions used: 2024-04-23T01:44:52.265 INFO:teuthology.orchestra.run.smithi113.stderr: count bytes template 2024-04-23T01:44:52.265 INFO:teuthology.orchestra.run.smithi113.stderr: 1 8 MallocExtension::Register 2024-04-23T01:44:52.265 INFO:teuthology.orchestra.run.smithi113.stderr:----------------------------------------------------- 2024-04-23T01:44:52.265 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-04-23T01:44:52.265 INFO:teuthology.orchestra.run.smithi113.stdout:created object store /var/lib/ceph/osd/ceph-2 for osd.2 fsid 0a731bfd-0fc4-4c1b-8e4b-f6dfbc059bcc 2024-04-23T01:44:52.287 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-23T01:44:52.311 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T01:44:52.595 INFO:teuthology.orchestra.run.smithi113.stderr:WARNING: debug mode. Not for benchmarking or production 2024-04-23T01:44:52.596 INFO:teuthology.orchestra.run.smithi113.stderr:INFO 2024-04-23 01:44:52,599 seastar - Reactor backend: linux-aio 2024-04-23T01:44:52.601 INFO:teuthology.orchestra.run.smithi113.stderr:==26506==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-04-23T01:44:52.713 INFO:teuthology.orchestra.run.smithi113.stderr:INFO 2024-04-23 01:44:52,715 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-04-23T01:44:52.713 INFO:teuthology.orchestra.run.smithi113.stderr:INFO 2024-04-23 01:44:52,715 [shard 0:main] osd - get_early_config: set --cpuset 3-5 2024-04-23T01:44:52.714 INFO:teuthology.orchestra.run.smithi113.stderr:DEBUG 2024-04-23 01:44:52,716 [shard 0:main] seastar - reactor::drain 2024-04-23T01:44:53.243 INFO:teuthology.orchestra.run.smithi113.stderr:----------------------------------------------------- 2024-04-23T01:44:53.243 INFO:teuthology.orchestra.run.smithi113.stderr:Suppressions used: 2024-04-23T01:44:53.243 INFO:teuthology.orchestra.run.smithi113.stderr: count bytes template 2024-04-23T01:44:53.243 INFO:teuthology.orchestra.run.smithi113.stderr: 1 8 MallocExtension::Register 2024-04-23T01:44:53.243 INFO:teuthology.orchestra.run.smithi113.stderr:----------------------------------------------------- 2024-04-23T01:44:53.243 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-04-23T01:44:53.301 INFO:teuthology.orchestra.run.smithi113.stderr:WARNING: debug mode. Not for benchmarking or production 2024-04-23T01:44:53.305 INFO:teuthology.orchestra.run.smithi113.stderr:INFO 2024-04-23 01:44:53,307 seastar - Reactor backend: linux-aio 2024-04-23T01:44:53.320 INFO:teuthology.orchestra.run.smithi113.stderr:==26504==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-04-23T01:44:57.942 INFO:teuthology.orchestra.run.smithi113.stderr:----------------------------------------------------- 2024-04-23T01:44:57.942 INFO:teuthology.orchestra.run.smithi113.stderr:Suppressions used: 2024-04-23T01:44:57.942 INFO:teuthology.orchestra.run.smithi113.stderr: count bytes template 2024-04-23T01:44:57.942 INFO:teuthology.orchestra.run.smithi113.stderr: 1 8 MallocExtension::Register 2024-04-23T01:44:57.942 INFO:teuthology.orchestra.run.smithi113.stderr:----------------------------------------------------- 2024-04-23T01:44:57.942 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-04-23T01:44:57.943 INFO:teuthology.orchestra.run.smithi113.stdout:created object store /var/lib/ceph/osd/ceph-3 for osd.3 fsid 0a731bfd-0fc4-4c1b-8e4b-f6dfbc059bcc 2024-04-23T01:44:57.963 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-23T01:44:57.986 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-23T01:44:57.986 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-23T01:44:57.986 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-23T01:44:58.015 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-23T01:44:58.015 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-23T01:44:58.083 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-23T01:44:58.083 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-23T01:44:58.106 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-23T01:44:58.106 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-23T01:44:58.169 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-23T01:44:58.169 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-23T01:44:58.232 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-23T01:44:58.233 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-23T01:44:58.250 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-23T01:44:58.250 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-23T01:44:58.287 INFO:tasks.ceph:Adding keys to all mons... 2024-04-23T01:44:58.288 DEBUG:teuthology.orchestra.run.smithi090:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-23T01:44:58.315 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.0] 2024-04-23T01:44:58.315 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQCCEidmh5VPNRAAFu7FL5599Z7+d+bd9kIt8Q== 2024-04-23T01:44:58.315 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.1] 2024-04-23T01:44:58.315 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQCIEidmu3ShJBAAiL3PMwUTUyHPgVu8gkmiVQ== 2024-04-23T01:44:58.315 INFO:teuthology.orchestra.run.smithi090.stdout:[mgr.x] 2024-04-23T01:44:58.315 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQCAEidmP3kHBRAAikwIHdfGpHAprkyFU3rtvg== 2024-04-23T01:44:58.315 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.2] 2024-04-23T01:44:58.315 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQCPEidm/jipLhAALTekg7ly+dG4Rb4/NQcvxQ== 2024-04-23T01:44:58.315 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.3] 2024-04-23T01:44:58.315 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQCVEidmdAwtGxAAnzhmaCXS2EcjDYn5iuXddQ== 2024-04-23T01:44:58.315 INFO:teuthology.orchestra.run.smithi090.stdout:[client.0] 2024-04-23T01:44:58.315 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQCAEidmalcYCBAAosBowvECZalfGnMAPnkpig== 2024-04-23T01:44:58.316 INFO:teuthology.orchestra.run.smithi090.stdout:[client.1] 2024-04-23T01:44:58.316 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQCAEidmXwSgCxAA2OaJa+W93wMI6bDljaSKhQ== 2024-04-23T01:44:58.316 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T01:44:58.419 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T01:44:58.478 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-23T01:44:58.536 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T01:44:58.594 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T01:44:58.654 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T01:44:58.715 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T01:44:58.776 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-23T01:44:58.776 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-23T01:44:58.799 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-23T01:44:58.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:44:58.908+0000 7f98d2430a80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:44:58.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:44:58.908+0000 7f98d2430a80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:44:58.921 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-23T01:44:58.944 DEBUG:teuthology.orchestra.run.smithi090:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-23T01:44:58.999 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-23T01:44:58.999 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-23T01:44:58.999 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-23T01:44:59.042 INFO:tasks.ceph.mon.a:Started 2024-04-23T01:44:59.042 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-23T01:44:59.043 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-23T01:44:59.043 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-23T01:44:59.046 INFO:tasks.ceph.mgr.x:Started 2024-04-23T01:44:59.046 INFO:tasks.ceph:Setting crush tunables to default 2024-04-23T01:44:59.046 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --cluster ceph osd crush tunables default 2024-04-23T01:44:59.122 INFO:tasks.ceph.mon.a.smithi090.stderr:2024-04-23T01:44:59.123+0000 7facff86fa80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:44:59.122 INFO:tasks.ceph.mon.a.smithi090.stderr:2024-04-23T01:44:59.123+0000 7facff86fa80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:44:59.122 INFO:tasks.ceph.mon.a.smithi090.stderr:2024-04-23T01:44:59.123+0000 7facff86fa80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:44:59.131 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-23T01:44:59.132+0000 7f9c1e22f000 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:44:59.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:44:59.149+0000 7f23092ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:44:59.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:44:59.160+0000 7f23092ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:44:59.229 INFO:teuthology.orchestra.run.smithi090.stderr:adjusted tunables profile to default 2024-04-23T01:44:59.240 INFO:tasks.ceph:check_enable_crimson: True 2024-04-23T01:44:59.240 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2024-04-23T01:44:59.241 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2024-04-23T01:44:59.336 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-23T01:44:59.338+0000 7f9c1e22f000 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:44:59.337 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-23T01:44:59.338+0000 7f9c1e22f000 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:44:59.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:44:59.341+0000 7fa99b06d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:44:59.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:44:59.349+0000 7fa99b06d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:44:59.427 INFO:tasks.ceph.mon.a.smithi090.stderr:2024-04-23T01:44:59.429+0000 7facf65f4640 -1 WARNING: experimental feature 'crimson' is enabled 2024-04-23T01:44:59.428 INFO:tasks.ceph.mon.a.smithi090.stderr:Please be aware that this feature is experimental, untested, 2024-04-23T01:44:59.428 INFO:tasks.ceph.mon.a.smithi090.stderr:unsupported, and may result in data corruption, data loss, 2024-04-23T01:44:59.428 INFO:tasks.ceph.mon.a.smithi090.stderr:and/or irreparable damage to your cluster. Do not use 2024-04-23T01:44:59.428 INFO:tasks.ceph.mon.a.smithi090.stderr:feature with important data. 2024-04-23T01:44:59.428 INFO:tasks.ceph.mon.a.smithi090.stderr: 2024-04-23T01:44:59.437 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-23T01:44:59.439+0000 7f9c1e22f000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T01:44:59.588 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-23T01:44:59.589+0000 7f9c1e22f000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T01:44:59.640 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-23T01:44:59.641+0000 7f9c1e22f000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T01:44:59.887 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-23T01:44:59.889+0000 7f9c1e22f000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T01:44:59.933 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-23T01:44:59.934+0000 7f9c1e22f000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T01:45:00.242 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-23T01:45:00.243 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-23T01:45:00.243 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-23T01:45:00.267 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-23T01:45:00.267 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-23T01:45:00.331 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-23T01:45:00.331 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-23T01:45:00.354 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-23T01:45:00.354 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-23T01:45:00.417 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 93e81b79-2e2e-4e90-ac13-13a26b240245 0 2024-04-23T01:45:00.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-23T01:45:00.559+0000 7f74bae6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:00.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-23T01:45:00.568+0000 7f74bae6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:00.646 INFO:teuthology.orchestra.run.smithi113.stdout:0 2024-04-23T01:45:00.657 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new e81e8b79-a326-41e5-9f97-1da36297e891 1 2024-04-23T01:45:00.679 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-23T01:45:00.680+0000 7f9c1e22f000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T01:45:00.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-23T01:45:00.757+0000 7f9909c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:00.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-23T01:45:00.766+0000 7f9909c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:00.770 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-23T01:45:00.772+0000 7f9c1e22f000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T01:45:00.814 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-23T01:45:00.816+0000 7f9c1e22f000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T01:45:00.843 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-04-23T01:45:00.854 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new a798cb89-96c0-4222-b32e-d08a97970e66 2 2024-04-23T01:45:00.989 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-23T01:45:00.990+0000 7f9c1e22f000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T01:45:01.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-23T01:45:00.996+0000 7f161d6ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:01.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-23T01:45:01.005+0000 7f161d6ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:01.044 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-23T01:45:01.045+0000 7f9c1e22f000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T01:45:01.083 INFO:teuthology.orchestra.run.smithi113.stdout:2 2024-04-23T01:45:01.095 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 446df045-a76b-4937-ae3f-dc84a4e1ca87 3 2024-04-23T01:45:01.098 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-23T01:45:01.100+0000 7f9c1e22f000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T01:45:01.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-23T01:45:01.196+0000 7fd5936ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:01.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-23T01:45:01.205+0000 7fd5936ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:01.218 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-23T01:45:01.220+0000 7f9c1e22f000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T01:45:01.283 INFO:teuthology.orchestra.run.smithi113.stdout:3 2024-04-23T01:45:01.294 DEBUG:teuthology.orchestra.run.smithi090:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2024-04-23T01:45:01.325 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-23T01:45:01.325 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2024-04-23T01:45:01.368 INFO:tasks.ceph.osd.0:Started 2024-04-23T01:45:01.368 DEBUG:teuthology.orchestra.run.smithi090:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2024-04-23T01:45:01.396 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-23T01:45:01.396 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-04-23T01:45:01.439 INFO:tasks.ceph.osd.1:Started 2024-04-23T01:45:01.439 DEBUG:teuthology.orchestra.run.smithi113:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2024-04-23T01:45:01.468 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-23T01:45:01.468 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2024-04-23T01:45:01.511 INFO:tasks.ceph.osd.2:Started 2024-04-23T01:45:01.511 DEBUG:teuthology.orchestra.run.smithi113:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.3.log 2024-04-23T01:45:01.538 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-23T01:45:01.538 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2024-04-23T01:45:01.581 INFO:tasks.ceph.osd.3:Started 2024-04-23T01:45:01.581 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T01:45:01.613 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-23T01:45:01.615+0000 7f9c1e22f000 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T01:45:01.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:01.692+0000 7fbe6495d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:01.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:01.700+0000 7fbe6495d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:01.768 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:45:01.769 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":7,"fsid":"0a731bfd-0fc4-4c1b-8e4b-f6dfbc059bcc","created":"2024-04-23T01:44:59.145590+0000","modified":"2024-04-23T01:45:01.284163+0000","last_up_change":"0.000000","last_in_change":"2024-04-23T01:45:01.284163+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"93e81b79-2e2e-4e90-ac13-13a26b240245","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e81e8b79-a326-41e5-9f97-1da36297e891","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a798cb89-96c0-4222-b32e-d08a97970e66","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"446df045-a76b-4937-ae3f-dc84a4e1ca87","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T01:45:01.780 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-23T01:45:01.780 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-23T01:45:01.899 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-23T01:45:01.900+0000 7f9c1e22f000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T01:45:01.953 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-23T01:45:01.954+0000 7f9c1e22f000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T01:45:02.074 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-23T01:45:02.075+0000 7f9c1e22f000 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T01:45:02.183 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-23T01:45:02.287 INFO:teuthology.misc.health.smithi090.stderr:2024-04-23T01:45:02.284+0000 7facfba6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:02.291 INFO:teuthology.misc.health.smithi090.stderr:2024-04-23T01:45:02.292+0000 7facfba6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:02.368 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-23T01:45:02.369+0000 7f9c1e22f000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T01:45:02.413 INFO:teuthology.misc.health.smithi090.stdout: 2024-04-23T01:45:02.414 INFO:teuthology.misc.health.smithi090.stdout:{"epoch":7,"fsid":"0a731bfd-0fc4-4c1b-8e4b-f6dfbc059bcc","created":"2024-04-23T01:44:59.145590+0000","modified":"2024-04-23T01:45:01.284163+0000","last_up_change":"0.000000","last_in_change":"2024-04-23T01:45:01.284163+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"93e81b79-2e2e-4e90-ac13-13a26b240245","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e81e8b79-a326-41e5-9f97-1da36297e891","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a798cb89-96c0-4222-b32e-d08a97970e66","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"446df045-a76b-4937-ae3f-dc84a4e1ca87","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T01:45:02.418 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-23T01:45:02.419+0000 7f9c1e22f000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T01:45:02.427 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-04-23T01:45:02.583 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-23T01:45:02.584+0000 7f9c1e22f000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T01:45:02.712 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-23T01:45:02.713+0000 7f9c1e22f000 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T01:45:02.762 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-23T01:45:02.763+0000 7f9c1e22f000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T01:45:02.832 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-23T01:45:02.834+0000 7f9c1e22f000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T01:45:02.879 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-23T01:45:02.880+0000 7f9c1e22f000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T01:45:03.418 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-23T01:45:03.419+0000 7f9c1e22f000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T01:45:03.537 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-23T01:45:03.538+0000 7f9c1e22f000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T01:45:04.550 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-23T01:45:04.551+0000 7f9bbd03e640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-04-23T01:45:08.831 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-23T01:45:08.936 INFO:teuthology.misc.health.smithi090.stderr:2024-04-23T01:45:08.933+0000 7f82cbe2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:08.940 INFO:teuthology.misc.health.smithi090.stderr:2024-04-23T01:45:08.941+0000 7f82cbe2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:09.134 INFO:teuthology.misc.health.smithi090.stdout: 2024-04-23T01:45:09.134 INFO:teuthology.misc.health.smithi090.stdout:{"epoch":11,"fsid":"0a731bfd-0fc4-4c1b-8e4b-f6dfbc059bcc","created":"2024-04-23T01:44:59.145590+0000","modified":"2024-04-23T01:45:08.616075+0000","last_up_change":"2024-04-23T01:45:06.551915+0000","last_in_change":"2024-04-23T01:45:01.284163+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T01:45:07.564959+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e81b79-2e2e-4e90-ac13-13a26b240245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6801","nonce":4128231813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":4128231813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6807","nonce":4128231813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":4128231813}]},"public_addr":"172.21.15.90:6801/4128231813","cluster_addr":"172.21.15.90:6802/4128231813","heartbeat_back_addr":"172.21.15.90:6807/4128231813","heartbeat_front_addr":"172.21.15.90:6806/4128231813","state":["exists","up"]},{"osd":1,"uuid":"e81e8b79-a326-41e5-9f97-1da36297e891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6803","nonce":55615551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":55615551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":55615551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":55615551}]},"public_addr":"172.21.15.90:6803/55615551","cluster_addr":"172.21.15.90:6804/55615551","heartbeat_back_addr":"172.21.15.90:6805/55615551","heartbeat_front_addr":"172.21.15.90:6800/55615551","state":["exists","up"]},{"osd":2,"uuid":"a798cb89-96c0-4222-b32e-d08a97970e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":474973433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":474973433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6811","nonce":474973433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":474973433}]},"public_addr":"172.21.15.113:6807/474973433","cluster_addr":"172.21.15.113:6806/474973433","heartbeat_back_addr":"172.21.15.113:6811/474973433","heartbeat_front_addr":"172.21.15.113:6810/474973433","state":["exists","up"]},{"osd":3,"uuid":"446df045-a76b-4937-ae3f-dc84a4e1ca87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6805","nonce":2075119886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":2075119886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":2075119886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6803","nonce":2075119886}]},"public_addr":"172.21.15.113:6805/2075119886","cluster_addr":"172.21.15.113:6804/2075119886","heartbeat_back_addr":"172.21.15.113:6808/2075119886","heartbeat_front_addr":"172.21.15.113:6803/2075119886","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T01:45:09.146 DEBUG:teuthology.misc:4 of 4 OSDs are up 2024-04-23T01:45:09.146 INFO:tasks.ceph:Creating RBD pool 2024-04-23T01:45:09.146 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --cluster ceph osd pool create rbd 8 2024-04-23T01:45:09.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:09.251+0000 7f62e595d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:09.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:09.262+0000 7f62e595d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:09.672 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'rbd' created 2024-04-23T01:45:09.688 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-04-23T01:45:09.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:09.805+0000 7f0ca8c2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:09.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:09.820+0000 7f0ca8c2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:10.677 INFO:teuthology.orchestra.run.smithi090.stderr:enabled application 'rbd' on pool 'rbd' 2024-04-23T01:45:10.699 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-23T01:45:10.700 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-23T01:45:10.700 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-23T01:45:10.700 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-23T01:45:10.700 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-23T01:45:10.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:10.851+0000 7fae6e31d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:10.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:10.858+0000 7fae6e31d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:11.096 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:45:11.107 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3221443217},{"type":"v1","addr":"172.21.15.113:6801","nonce":3221443217}]},"active_addr":"172.21.15.113:6801/3221443217","active_change":"2024-04-23T01:45:03.541319+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.113:0","nonce":3190851598}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.113:0","nonce":4165234019}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.113:0","nonce":1830404768}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.113:0","nonce":3141221239}]}]} 2024-04-23T01:45:11.109 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-23T01:45:11.109 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-23T01:45:11.110 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T01:45:11.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:11.214+0000 7fe52e99d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:11.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:11.221+0000 7fe52e99d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:11.410 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:45:11.410 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":13,"fsid":"0a731bfd-0fc4-4c1b-8e4b-f6dfbc059bcc","created":"2024-04-23T01:44:59.145590+0000","modified":"2024-04-23T01:45:10.675479+0000","last_up_change":"2024-04-23T01:45:06.551915+0000","last_in_change":"2024-04-23T01:45:01.284163+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T01:45:07.564959+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-23T01:45:09.460600+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e81b79-2e2e-4e90-ac13-13a26b240245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6801","nonce":4128231813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":4128231813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6807","nonce":4128231813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":4128231813}]},"public_addr":"172.21.15.90:6801/4128231813","cluster_addr":"172.21.15.90:6802/4128231813","heartbeat_back_addr":"172.21.15.90:6807/4128231813","heartbeat_front_addr":"172.21.15.90:6806/4128231813","state":["exists","up"]},{"osd":1,"uuid":"e81e8b79-a326-41e5-9f97-1da36297e891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6803","nonce":55615551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":55615551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":55615551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":55615551}]},"public_addr":"172.21.15.90:6803/55615551","cluster_addr":"172.21.15.90:6804/55615551","heartbeat_back_addr":"172.21.15.90:6805/55615551","heartbeat_front_addr":"172.21.15.90:6800/55615551","state":["exists","up"]},{"osd":2,"uuid":"a798cb89-96c0-4222-b32e-d08a97970e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":474973433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":474973433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6811","nonce":474973433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":474973433}]},"public_addr":"172.21.15.113:6807/474973433","cluster_addr":"172.21.15.113:6806/474973433","heartbeat_back_addr":"172.21.15.113:6811/474973433","heartbeat_front_addr":"172.21.15.113:6810/474973433","state":["exists","up"]},{"osd":3,"uuid":"446df045-a76b-4937-ae3f-dc84a4e1ca87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6805","nonce":2075119886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":2075119886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":2075119886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6803","nonce":2075119886}]},"public_addr":"172.21.15.113:6805/2075119886","cluster_addr":"172.21.15.113:6804/2075119886","heartbeat_back_addr":"172.21.15.113:6808/2075119886","heartbeat_front_addr":"172.21.15.113:6803/2075119886","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T01:45:11.423 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-23T01:45:11.423 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T01:45:11.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:11.526+0000 7f8d1e42d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:11.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:11.533+0000 7f8d1e42d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:11.725 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:45:11.726 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":14,"fsid":"0a731bfd-0fc4-4c1b-8e4b-f6dfbc059bcc","created":"2024-04-23T01:44:59.145590+0000","modified":"2024-04-23T01:45:11.679701+0000","last_up_change":"2024-04-23T01:45:06.551915+0000","last_in_change":"2024-04-23T01:45:01.284163+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T01:45:07.564959+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-23T01:45:09.460600+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e81b79-2e2e-4e90-ac13-13a26b240245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6801","nonce":4128231813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":4128231813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6807","nonce":4128231813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":4128231813}]},"public_addr":"172.21.15.90:6801/4128231813","cluster_addr":"172.21.15.90:6802/4128231813","heartbeat_back_addr":"172.21.15.90:6807/4128231813","heartbeat_front_addr":"172.21.15.90:6806/4128231813","state":["exists","up"]},{"osd":1,"uuid":"e81e8b79-a326-41e5-9f97-1da36297e891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6803","nonce":55615551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":55615551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":55615551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":55615551}]},"public_addr":"172.21.15.90:6803/55615551","cluster_addr":"172.21.15.90:6804/55615551","heartbeat_back_addr":"172.21.15.90:6805/55615551","heartbeat_front_addr":"172.21.15.90:6800/55615551","state":["exists","up"]},{"osd":2,"uuid":"a798cb89-96c0-4222-b32e-d08a97970e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":474973433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":474973433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6811","nonce":474973433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":474973433}]},"public_addr":"172.21.15.113:6807/474973433","cluster_addr":"172.21.15.113:6806/474973433","heartbeat_back_addr":"172.21.15.113:6811/474973433","heartbeat_front_addr":"172.21.15.113:6810/474973433","state":["exists","up"]},{"osd":3,"uuid":"446df045-a76b-4937-ae3f-dc84a4e1ca87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6805","nonce":2075119886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":2075119886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":2075119886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6803","nonce":2075119886}]},"public_addr":"172.21.15.113:6805/2075119886","cluster_addr":"172.21.15.113:6804/2075119886","heartbeat_back_addr":"172.21.15.113:6808/2075119886","heartbeat_front_addr":"172.21.15.113:6803/2075119886","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T01:45:11.738 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-23T01:45:11.739 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-23T01:45:11.739 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-23T01:45:11.739 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-23T01:45:11.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:11.853+0000 7ff21f59d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:11.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:11.853+0000 7f42e442d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:11.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:11.853+0000 7f171fa20640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:11.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:11.853+0000 7fe3f282d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:11.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:11.863+0000 7ff21f59d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:11.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:11.863+0000 7f171fa20640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:11.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:11.863+0000 7f42e442d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:11.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:11.863+0000 7fe3f282d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:11.894 INFO:teuthology.orchestra.run.smithi090.stdout:38654705669 2024-04-23T01:45:11.895 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-23T01:45:11.898 INFO:teuthology.orchestra.run.smithi090.stdout:38654705669 2024-04-23T01:45:11.898 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-23T01:45:11.900 INFO:teuthology.orchestra.run.smithi090.stdout:38654705669 2024-04-23T01:45:11.900 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-23T01:45:11.901 INFO:teuthology.orchestra.run.smithi090.stdout:38654705669 2024-04-23T01:45:11.901 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-23T01:45:12.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:12.003+0000 7f29196ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:12.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:12.013+0000 7f29196ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:12.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:12.051+0000 7f5569220640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:12.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:12.052+0000 7ffbd62ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:12.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:12.060+0000 7f5569220640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:12.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:12.055+0000 7fde29d1d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:12.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:12.062+0000 7ffbd62ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:12.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:12.067+0000 7fde29d1d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:12.222 INFO:teuthology.orchestra.run.smithi090.stdout:38654705669 2024-04-23T01:45:12.236 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.1 2024-04-23T01:45:12.236 DEBUG:teuthology.parallel:result is None 2024-04-23T01:45:12.270 INFO:teuthology.orchestra.run.smithi090.stdout:38654705669 2024-04-23T01:45:12.273 INFO:teuthology.orchestra.run.smithi090.stdout:38654705669 2024-04-23T01:45:12.284 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.2 2024-04-23T01:45:12.284 DEBUG:teuthology.parallel:result is None 2024-04-23T01:45:12.287 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.3 2024-04-23T01:45:12.287 INFO:teuthology.orchestra.run.smithi090.stdout:38654705669 2024-04-23T01:45:12.287 DEBUG:teuthology.parallel:result is None 2024-04-23T01:45:12.300 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.0 2024-04-23T01:45:12.300 DEBUG:teuthology.parallel:result is None 2024-04-23T01:45:12.300 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-23T01:45:12.301 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:45:12.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:12.445+0000 7fdb24d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:12.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:12.452+0000 7fdb24d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:12.638 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:45:12.639 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:45:12.651 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-04-23T01:45:11.690805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":107944,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377379416,"statfs":{"total":386547056640,"available":386436521984,"internally_reserved":0,"allocated":626688,"data_stored":1880100,"data_compressed":89859,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.112068"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.778515+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:45:10.778515+0000","last_peered":"2024-04-23T01:45:10.778515+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:45:10.778515+0000","last_undegraded":"2024-04-23T01:45:10.778515+0000","last_fullsized":"2024-04-23T01:45:10.778515+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.768599+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:45:10.768599+0000","last_peered":"2024-04-23T01:45:10.768599+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:45:10.768599+0000","last_undegraded":"2024-04-23T01:45:10.768599+0000","last_fullsized":"2024-04-23T01:45:10.768599+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.760005+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:45:10.760005+0000","last_peered":"2024-04-23T01:45:10.760005+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:45:10.760005+0000","last_undegraded":"2024-04-23T01:45:10.760005+0000","last_fullsized":"2024-04-23T01:45:10.760005+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.765999+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:45:10.765999+0000","last_peered":"2024-04-23T01:45:10.765999+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:45:10.765999+0000","last_undegraded":"2024-04-23T01:45:10.765999+0000","last_fullsized":"2024-04-23T01:45:10.765999+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.782842+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:45:10.782842+0000","last_peered":"2024-04-23T01:45:10.782842+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:45:10.782842+0000","last_undegraded":"2024-04-23T01:45:10.782842+0000","last_fullsized":"2024-04-23T01:45:10.782842+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.753951+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:45:10.753951+0000","last_peered":"2024-04-23T01:45:10.753951+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:45:10.753951+0000","last_undegraded":"2024-04-23T01:45:10.753951+0000","last_fullsized":"2024-04-23T01:45:10.753951+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.770112+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:45:10.770112+0000","last_peered":"2024-04-23T01:45:10.770112+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:45:10.770112+0000","last_undegraded":"2024-04-23T01:45:10.770112+0000","last_fullsized":"2024-04-23T01:45:10.770112+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.803460+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:45:10.803460+0000","last_peered":"2024-04-23T01:45:10.803460+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:45:10.803460+0000","last_undegraded":"2024-04-23T01:45:10.803460+0000","last_fullsized":"2024-04-23T01:45:10.803460+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":142,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.691907+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:45:10.691907+0000","last_peered":"2024-04-23T01:45:10.691907+0000","last_clean":"2024-04-23T01:45:10.691907+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:45:10.691907+0000","last_undegraded":"2024-04-23T01:45:10.691907+0000","last_fullsized":"2024-04-23T01:45:10.691907+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27004,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344836,"statfs":{"total":96636764160,"available":96609112064,"internally_reserved":0,"allocated":172032,"data_stored":617617,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27004,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344836,"statfs":{"total":96636764160,"available":96609112064,"internally_reserved":0,"allocated":172032,"data_stored":617617,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26932,"kb_used_data":108,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344908,"statfs":{"total":96636764160,"available":96609185792,"internally_reserved":0,"allocated":110592,"data_stored":27249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27004,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344836,"statfs":{"total":96636764160,"available":96609112064,"internally_reserved":0,"allocated":172032,"data_stored":617617,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:45:12.652 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:45:12.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:12.756+0000 7f276c79d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:12.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:12.764+0000 7f276c79d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:12.946 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:45:12.947 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:45:12.958 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-04-23T01:45:11.690805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":107944,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377379416,"statfs":{"total":386547056640,"available":386436521984,"internally_reserved":0,"allocated":626688,"data_stored":1880100,"data_compressed":89859,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.112068"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.778515+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:45:10.778515+0000","last_peered":"2024-04-23T01:45:10.778515+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:45:10.778515+0000","last_undegraded":"2024-04-23T01:45:10.778515+0000","last_fullsized":"2024-04-23T01:45:10.778515+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.768599+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:45:10.768599+0000","last_peered":"2024-04-23T01:45:10.768599+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:45:10.768599+0000","last_undegraded":"2024-04-23T01:45:10.768599+0000","last_fullsized":"2024-04-23T01:45:10.768599+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.760005+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:45:10.760005+0000","last_peered":"2024-04-23T01:45:10.760005+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:45:10.760005+0000","last_undegraded":"2024-04-23T01:45:10.760005+0000","last_fullsized":"2024-04-23T01:45:10.760005+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.765999+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:45:10.765999+0000","last_peered":"2024-04-23T01:45:10.765999+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:45:10.765999+0000","last_undegraded":"2024-04-23T01:45:10.765999+0000","last_fullsized":"2024-04-23T01:45:10.765999+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.782842+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:45:10.782842+0000","last_peered":"2024-04-23T01:45:10.782842+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:45:10.782842+0000","last_undegraded":"2024-04-23T01:45:10.782842+0000","last_fullsized":"2024-04-23T01:45:10.782842+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.753951+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:45:10.753951+0000","last_peered":"2024-04-23T01:45:10.753951+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:45:10.753951+0000","last_undegraded":"2024-04-23T01:45:10.753951+0000","last_fullsized":"2024-04-23T01:45:10.753951+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.770112+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:45:10.770112+0000","last_peered":"2024-04-23T01:45:10.770112+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:45:10.770112+0000","last_undegraded":"2024-04-23T01:45:10.770112+0000","last_fullsized":"2024-04-23T01:45:10.770112+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.803460+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:45:10.803460+0000","last_peered":"2024-04-23T01:45:10.803460+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:45:10.803460+0000","last_undegraded":"2024-04-23T01:45:10.803460+0000","last_fullsized":"2024-04-23T01:45:10.803460+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":142,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.691907+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:45:10.691907+0000","last_peered":"2024-04-23T01:45:10.691907+0000","last_clean":"2024-04-23T01:45:10.691907+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:45:10.691907+0000","last_undegraded":"2024-04-23T01:45:10.691907+0000","last_fullsized":"2024-04-23T01:45:10.691907+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27004,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344836,"statfs":{"total":96636764160,"available":96609112064,"internally_reserved":0,"allocated":172032,"data_stored":617617,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27004,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344836,"statfs":{"total":96636764160,"available":96609112064,"internally_reserved":0,"allocated":172032,"data_stored":617617,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26932,"kb_used_data":108,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344908,"statfs":{"total":96636764160,"available":96609185792,"internally_reserved":0,"allocated":110592,"data_stored":27249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27004,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344836,"statfs":{"total":96636764160,"available":96609112064,"internally_reserved":0,"allocated":172032,"data_stored":617617,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:45:12.959 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-23T01:45:12.959 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-23T01:45:12.959 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-23T01:45:12.959 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-23T01:45:13.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:13.063+0000 7f5aebd9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:13.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:13.069+0000 7f5aebd9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:13.290 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:45:13.290 INFO:teuthology.orchestra.run.smithi090.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-23T01:45:13.304 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-23T01:45:13.304 INFO:teuthology.run_tasks:Running task thrashosds... 2024-04-23T01:45:13.316 INFO:tasks.thrashosds:config is {'ceph_objectstore_tool': False, 'chance_down': 0, 'chance_inject_pause_short': 0, 'chance_reset_purged_snaps_last': 0, 'chance_thrash_cluster_full': 0, 'chance_thrash_pg_upmap': 0, 'dump_ops_enable': False, 'min_in': 3, 'noscrub_toggle_delay': 0, 'reweight_osd': 0, 'sighup_delay': 0, 'thrash_primary_affinity': False, 'timeout': 2400, 'optrack_toggle_delay': 2.0, 'random_eio': 0.0} 2024-04-23T01:45:13.316 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'ci-yingxin-crimson-fix-tri-mutex-4'}, 'ceph': {'conf': {'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore rocksdb cf': False, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'err to stderr': True, 'log flush on exit': True, 'log to file': False, 'log to stderr': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'flavor': 'crimson', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum'], 'sha1': 'a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'crimson', 'sha1': 'a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'min_in': 4}, 'workunit': {'branch': 'ci-yingxin-crimson-fix-tri-mutex-4', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': 'a1f34943cf7f9ae8af14bde99c6ca6a06e193ce4'}} 2024-04-23T01:45:13.316 INFO:tasks.thrashosds:config is {'ceph_objectstore_tool': False, 'chance_down': 0, 'chance_inject_pause_short': 0, 'chance_reset_purged_snaps_last': 0, 'chance_thrash_cluster_full': 0, 'chance_thrash_pg_upmap': 0, 'dump_ops_enable': False, 'min_in': 4, 'noscrub_toggle_delay': 0, 'reweight_osd': 0, 'sighup_delay': 0, 'thrash_primary_affinity': False, 'timeout': 2400, 'optrack_toggle_delay': 2.0, 'random_eio': 0.0} 2024-04-23T01:45:13.316 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-04-23T01:45:13.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:13.420+0000 7f1b530ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:13.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:13.430+0000 7f1b530ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:13.632 INFO:tasks.thrashosds:Beginning thrashosds... 2024-04-23T01:45:13.632 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-23T01:45:13.633 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:45:13.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:13.737+0000 7fbf8515d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:13.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:13.745+0000 7fbf8515d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:13.927 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:45:13.928 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:45:13.940 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-04-23T01:45:13.548705+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":107944,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377379416,"statfs":{"total":386547056640,"available":386436521984,"internally_reserved":0,"allocated":626688,"data_stored":1880100,"data_compressed":89859,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.969968"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.778515+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:45:10.778515+0000","last_peered":"2024-04-23T01:45:10.778515+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:45:10.778515+0000","last_undegraded":"2024-04-23T01:45:10.778515+0000","last_fullsized":"2024-04-23T01:45:10.778515+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.768599+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:45:10.768599+0000","last_peered":"2024-04-23T01:45:10.768599+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:45:10.768599+0000","last_undegraded":"2024-04-23T01:45:10.768599+0000","last_fullsized":"2024-04-23T01:45:10.768599+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T01:45:11.703427+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:45:11.703427+0000","last_peered":"2024-04-23T01:45:11.703427+0000","last_clean":"2024-04-23T01:45:11.703427+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:45:11.703427+0000","last_undegraded":"2024-04-23T01:45:11.703427+0000","last_fullsized":"2024-04-23T01:45:11.703427+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.765999+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:45:10.765999+0000","last_peered":"2024-04-23T01:45:10.765999+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:45:10.765999+0000","last_undegraded":"2024-04-23T01:45:10.765999+0000","last_fullsized":"2024-04-23T01:45:10.765999+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.782842+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:45:10.782842+0000","last_peered":"2024-04-23T01:45:10.782842+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:45:10.782842+0000","last_undegraded":"2024-04-23T01:45:10.782842+0000","last_fullsized":"2024-04-23T01:45:10.782842+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.753951+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:45:10.753951+0000","last_peered":"2024-04-23T01:45:10.753951+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:45:10.753951+0000","last_undegraded":"2024-04-23T01:45:10.753951+0000","last_fullsized":"2024-04-23T01:45:10.753951+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.770112+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:45:10.770112+0000","last_peered":"2024-04-23T01:45:10.770112+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:45:10.770112+0000","last_undegraded":"2024-04-23T01:45:10.770112+0000","last_fullsized":"2024-04-23T01:45:10.770112+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.803460+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:45:10.803460+0000","last_peered":"2024-04-23T01:45:10.803460+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:45:10.803460+0000","last_undegraded":"2024-04-23T01:45:10.803460+0000","last_fullsized":"2024-04-23T01:45:10.803460+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":145,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T01:45:11.699272+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:45:11.699272+0000","last_peered":"2024-04-23T01:45:11.699272+0000","last_clean":"2024-04-23T01:45:11.699272+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:45:11.699272+0000","last_undegraded":"2024-04-23T01:45:11.699272+0000","last_fullsized":"2024-04-23T01:45:11.699272+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27004,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344836,"statfs":{"total":96636764160,"available":96609112064,"internally_reserved":0,"allocated":172032,"data_stored":617617,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27004,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344836,"statfs":{"total":96636764160,"available":96609112064,"internally_reserved":0,"allocated":172032,"data_stored":617617,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26932,"kb_used_data":108,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344908,"statfs":{"total":96636764160,"available":96609185792,"internally_reserved":0,"allocated":110592,"data_stored":27249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27004,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344836,"statfs":{"total":96636764160,"available":96609112064,"internally_reserved":0,"allocated":172032,"data_stored":617617,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:45:13.940 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:45:14.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:14.043+0000 7f410d46d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:14.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:14.051+0000 7f410d46d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:14.235 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:45:14.235 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:45:14.247 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-04-23T01:45:13.548705+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":107944,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377379416,"statfs":{"total":386547056640,"available":386436521984,"internally_reserved":0,"allocated":626688,"data_stored":1880100,"data_compressed":89859,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.969968"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.778515+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:45:10.778515+0000","last_peered":"2024-04-23T01:45:10.778515+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:45:10.778515+0000","last_undegraded":"2024-04-23T01:45:10.778515+0000","last_fullsized":"2024-04-23T01:45:10.778515+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.768599+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:45:10.768599+0000","last_peered":"2024-04-23T01:45:10.768599+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:45:10.768599+0000","last_undegraded":"2024-04-23T01:45:10.768599+0000","last_fullsized":"2024-04-23T01:45:10.768599+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T01:45:11.703427+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:45:11.703427+0000","last_peered":"2024-04-23T01:45:11.703427+0000","last_clean":"2024-04-23T01:45:11.703427+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:45:11.703427+0000","last_undegraded":"2024-04-23T01:45:11.703427+0000","last_fullsized":"2024-04-23T01:45:11.703427+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.765999+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:45:10.765999+0000","last_peered":"2024-04-23T01:45:10.765999+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:45:10.765999+0000","last_undegraded":"2024-04-23T01:45:10.765999+0000","last_fullsized":"2024-04-23T01:45:10.765999+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.782842+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:45:10.782842+0000","last_peered":"2024-04-23T01:45:10.782842+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:45:10.782842+0000","last_undegraded":"2024-04-23T01:45:10.782842+0000","last_fullsized":"2024-04-23T01:45:10.782842+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.753951+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:45:10.753951+0000","last_peered":"2024-04-23T01:45:10.753951+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:45:10.753951+0000","last_undegraded":"2024-04-23T01:45:10.753951+0000","last_fullsized":"2024-04-23T01:45:10.753951+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.770112+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:45:10.770112+0000","last_peered":"2024-04-23T01:45:10.770112+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:45:10.770112+0000","last_undegraded":"2024-04-23T01:45:10.770112+0000","last_fullsized":"2024-04-23T01:45:10.770112+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.803460+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:45:10.803460+0000","last_peered":"2024-04-23T01:45:10.803460+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:45:10.803460+0000","last_undegraded":"2024-04-23T01:45:10.803460+0000","last_fullsized":"2024-04-23T01:45:10.803460+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":145,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T01:45:11.699272+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:45:11.699272+0000","last_peered":"2024-04-23T01:45:11.699272+0000","last_clean":"2024-04-23T01:45:11.699272+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:45:11.699272+0000","last_undegraded":"2024-04-23T01:45:11.699272+0000","last_fullsized":"2024-04-23T01:45:11.699272+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27004,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344836,"statfs":{"total":96636764160,"available":96609112064,"internally_reserved":0,"allocated":172032,"data_stored":617617,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27004,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344836,"statfs":{"total":96636764160,"available":96609112064,"internally_reserved":0,"allocated":172032,"data_stored":617617,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26932,"kb_used_data":108,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344908,"statfs":{"total":96636764160,"available":96609185792,"internally_reserved":0,"allocated":110592,"data_stored":27249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27004,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344836,"statfs":{"total":96636764160,"available":96609112064,"internally_reserved":0,"allocated":172032,"data_stored":617617,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:45:14.248 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-23T01:45:14.248 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-04-23T01:45:14.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:14.352+0000 7f680075d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:14.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:14.359+0000 7f680075d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:14.544 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 14 2024-04-23T01:45:14.544 INFO:teuthology.orchestra.run.smithi090.stdout:fsid 0a731bfd-0fc4-4c1b-8e4b-f6dfbc059bcc 2024-04-23T01:45:14.544 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-23T01:44:59.145590+0000 2024-04-23T01:45:14.544 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-23T01:45:11.679701+0000 2024-04-23T01:45:14.544 INFO:teuthology.orchestra.run.smithi090.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-23T01:45:14.544 INFO:teuthology.orchestra.run.smithi090.stdout:crush_version 3 2024-04-23T01:45:14.545 INFO:teuthology.orchestra.run.smithi090.stdout:full_ratio 0.95 2024-04-23T01:45:14.545 INFO:teuthology.orchestra.run.smithi090.stdout:backfillfull_ratio 0.9 2024-04-23T01:45:14.545 INFO:teuthology.orchestra.run.smithi090.stdout:nearfull_ratio 0.85 2024-04-23T01:45:14.545 INFO:teuthology.orchestra.run.smithi090.stdout:require_min_compat_client luminous 2024-04-23T01:45:14.545 INFO:teuthology.orchestra.run.smithi090.stdout:min_compat_client jewel 2024-04-23T01:45:14.545 INFO:teuthology.orchestra.run.smithi090.stdout:require_osd_release squid 2024-04-23T01:45:14.545 INFO:teuthology.orchestra.run.smithi090.stdout:stretch_mode_enabled false 2024-04-23T01:45:14.545 INFO:teuthology.orchestra.run.smithi090.stdout:allow_crimson=true 2024-04-23T01:45:14.545 INFO:teuthology.orchestra.run.smithi090.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 12 flags hashpspool,nopgchange,crimson stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 4.00 2024-04-23T01:45:14.545 INFO:teuthology.orchestra.run.smithi090.stdout:pool 2 'rbd' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 14 flags hashpspool,nopgchange,crimson stripe_width 0 application rbd read_balance_score 1.50 2024-04-23T01:45:14.545 INFO:teuthology.orchestra.run.smithi090.stdout:max_osd 4 2024-04-23T01:45:14.545 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 up in weight 1 up_from 9 up_thru 12 down_at 0 last_clean_interval [0,0) v2:172.21.15.90:6801/4128231813 v2:172.21.15.90:6802/4128231813 exists,up 93e81b79-2e2e-4e90-ac13-13a26b240245 2024-04-23T01:45:14.545 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 up in weight 1 up_from 9 up_thru 12 down_at 0 last_clean_interval [0,0) v2:172.21.15.90:6803/55615551 v2:172.21.15.90:6804/55615551 exists,up e81e8b79-a326-41e5-9f97-1da36297e891 2024-04-23T01:45:14.545 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 up in weight 1 up_from 9 up_thru 12 down_at 0 last_clean_interval [0,0) v2:172.21.15.113:6807/474973433 v2:172.21.15.113:6806/474973433 exists,up a798cb89-96c0-4222-b32e-d08a97970e66 2024-04-23T01:45:14.546 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 up in weight 1 up_from 9 up_thru 12 down_at 0 last_clean_interval [0,0) v2:172.21.15.113:6805/2075119886 v2:172.21.15.113:6804/2075119886 exists,up 446df045-a76b-4937-ae3f-dc84a4e1ca87 2024-04-23T01:45:14.557 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 9 up_thru 12 down_at 0 last_clean_interval [0,0) v2:172.21.15.90:6801/4128231813 v2:172.21.15.90:6802/4128231813 exists,up 93e81b79-2e2e-4e90-ac13-13a26b240245', 'osd.1 up in weight 1 up_from 9 up_thru 12 down_at 0 last_clean_interval [0,0) v2:172.21.15.90:6803/55615551 v2:172.21.15.90:6804/55615551 exists,up e81e8b79-a326-41e5-9f97-1da36297e891', 'osd.2 up in weight 1 up_from 9 up_thru 12 down_at 0 last_clean_interval [0,0) v2:172.21.15.113:6807/474973433 v2:172.21.15.113:6806/474973433 exists,up a798cb89-96c0-4222-b32e-d08a97970e66', 'osd.3 up in weight 1 up_from 9 up_thru 12 down_at 0 last_clean_interval [0,0) v2:172.21.15.113:6805/2075119886 v2:172.21.15.113:6804/2075119886 exists,up 446df045-a76b-4937-ae3f-dc84a4e1ca87'] 2024-04-23T01:45:14.557 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-04-23T01:45:14.662 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-23T01:45:14.662 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mon.a", 2024-04-23T01:45:14.662 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster": "ceph", 2024-04-23T01:45:14.662 INFO:teuthology.orchestra.run.smithi090.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-04-23T01:45:14.662 INFO:teuthology.orchestra.run.smithi090.stdout: "admin_socket_mode": "", 2024-04-23T01:45:14.662 INFO:teuthology.orchestra.run.smithi090.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-04-23T01:45:14.663 INFO:teuthology.orchestra.run.smithi090.stdout: "auth_client_required": "cephx, none", 2024-04-23T01:45:14.663 INFO:teuthology.orchestra.run.smithi090.stdout: "auth_cluster_required": "cephx", 2024-04-23T01:45:14.663 INFO:teuthology.orchestra.run.smithi090.stdout: "auth_debug": "true", 2024-04-23T01:45:14.663 INFO:teuthology.orchestra.run.smithi090.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-04-23T01:45:14.663 INFO:teuthology.orchestra.run.smithi090.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-04-23T01:45:14.663 INFO:teuthology.orchestra.run.smithi090.stdout: "auth_service_required": "cephx", 2024-04-23T01:45:14.663 INFO:teuthology.orchestra.run.smithi090.stdout: "auth_service_ticket_ttl": "240.000000", 2024-04-23T01:45:14.663 INFO:teuthology.orchestra.run.smithi090.stdout: "auth_supported": "cephx", 2024-04-23T01:45:14.663 INFO:teuthology.orchestra.run.smithi090.stdout: "bdev_aio": "true", 2024-04-23T01:45:14.663 INFO:teuthology.orchestra.run.smithi090.stdout: "bdev_aio_max_queue_depth": "1024", 2024-04-23T01:45:14.664 INFO:teuthology.orchestra.run.smithi090.stdout: "bdev_aio_poll_ms": "250", 2024-04-23T01:45:14.664 INFO:teuthology.orchestra.run.smithi090.stdout: "bdev_aio_reap_max": "16", 2024-04-23T01:45:14.664 INFO:teuthology.orchestra.run.smithi090.stdout: "bdev_async_discard_threads": "0", 2024-04-23T01:45:14.664 INFO:teuthology.orchestra.run.smithi090.stdout: "bdev_block_size": "4096", 2024-04-23T01:45:14.664 INFO:teuthology.orchestra.run.smithi090.stdout: "bdev_debug_aio": "false", 2024-04-23T01:45:14.664 INFO:teuthology.orchestra.run.smithi090.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-04-23T01:45:14.664 INFO:teuthology.orchestra.run.smithi090.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-04-23T01:45:14.664 INFO:teuthology.orchestra.run.smithi090.stdout: "bdev_debug_inflight_ios": "false", 2024-04-23T01:45:14.664 INFO:teuthology.orchestra.run.smithi090.stdout: "bdev_enable_discard": "false", 2024-04-23T01:45:14.664 INFO:teuthology.orchestra.run.smithi090.stdout: "bdev_flock_retry": "3", 2024-04-23T01:45:14.664 INFO:teuthology.orchestra.run.smithi090.stdout: "bdev_flock_retry_interval": "0.100000", 2024-04-23T01:45:14.664 INFO:teuthology.orchestra.run.smithi090.stdout: "bdev_inject_crash": "0", 2024-04-23T01:45:14.664 INFO:teuthology.orchestra.run.smithi090.stdout: "bdev_inject_crash_flush_delay": "2", 2024-04-23T01:45:14.665 INFO:teuthology.orchestra.run.smithi090.stdout: "bdev_ioring": "false", 2024-04-23T01:45:14.665 INFO:teuthology.orchestra.run.smithi090.stdout: "bdev_ioring_hipri": "false", 2024-04-23T01:45:14.665 INFO:teuthology.orchestra.run.smithi090.stdout: "bdev_ioring_sqthread_poll": "false", 2024-04-23T01:45:14.665 INFO:teuthology.orchestra.run.smithi090.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-04-23T01:45:14.665 INFO:teuthology.orchestra.run.smithi090.stdout: "bdev_read_buffer_alignment": "4096", 2024-04-23T01:45:14.665 INFO:teuthology.orchestra.run.smithi090.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-04-23T01:45:14.665 INFO:teuthology.orchestra.run.smithi090.stdout: "bdev_type": "", 2024-04-23T01:45:14.665 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_alloc_size": "1048576", 2024-04-23T01:45:14.665 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_allocator": "hybrid", 2024-04-23T01:45:14.665 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_buffered_io": "true", 2024-04-23T01:45:14.665 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_check_for_zeros": "false", 2024-04-23T01:45:14.665 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_check_volume_selector_often": "false", 2024-04-23T01:45:14.665 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-04-23T01:45:14.665 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_compact_log_sync": "false", 2024-04-23T01:45:14.665 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-04-23T01:45:14.666 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-04-23T01:45:14.666 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_log_compact_min_size": "16777216", 2024-04-23T01:45:14.666 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_log_replay_check_allocations": "true", 2024-04-23T01:45:14.666 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_max_log_runway": "4194304", 2024-04-23T01:45:14.666 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_max_prefetch": "1048576", 2024-04-23T01:45:14.666 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_min_flush_size": "524288", 2024-04-23T01:45:14.666 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_min_log_runway": "1048576", 2024-04-23T01:45:14.666 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_replay_recovery": "false", 2024-04-23T01:45:14.666 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-04-23T01:45:14.666 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_shared_alloc_size": "65536", 2024-04-23T01:45:14.666 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_sync_write": "false", 2024-04-23T01:45:14.666 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-04-23T01:45:14.666 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-04-23T01:45:14.666 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-04-23T01:45:14.667 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_allocation_from_file": "true", 2024-04-23T01:45:14.667 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_allocator": "hybrid", 2024-04-23T01:45:14.667 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-04-23T01:45:14.667 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-04-23T01:45:14.667 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-04-23T01:45:14.667 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-04-23T01:45:14.667 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-04-23T01:45:14.667 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-04-23T01:45:14.667 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blobid_prealloc": "10240", 2024-04-23T01:45:14.667 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_block_create": "true", 2024-04-23T01:45:14.667 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_block_db_create": "false", 2024-04-23T01:45:14.667 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_block_db_path": "", 2024-04-23T01:45:14.667 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_block_db_size": "0", 2024-04-23T01:45:14.667 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_block_path": "", 2024-04-23T01:45:14.668 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_block_preallocate_file": "false", 2024-04-23T01:45:14.668 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_block_size": "107374182400", 2024-04-23T01:45:14.668 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_block_wal_create": "false", 2024-04-23T01:45:14.668 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_block_wal_path": "", 2024-04-23T01:45:14.668 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_block_wal_size": "100663296", 2024-04-23T01:45:14.668 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_bluefs": "true", 2024-04-23T01:45:14.668 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-04-23T01:45:14.668 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_bluefs_env_mirror": "false", 2024-04-23T01:45:14.668 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-04-23T01:45:14.668 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-04-23T01:45:14.668 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-04-23T01:45:14.668 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-04-23T01:45:14.668 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-04-23T01:45:14.668 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-04-23T01:45:14.669 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_autotune": "true", 2024-04-23T01:45:14.669 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-04-23T01:45:14.669 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-04-23T01:45:14.669 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-04-23T01:45:14.669 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-04-23T01:45:14.669 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_size": "0", 2024-04-23T01:45:14.669 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-04-23T01:45:14.669 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-04-23T01:45:14.669 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-04-23T01:45:14.669 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-04-23T01:45:14.669 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_type": "2q", 2024-04-23T01:45:14.669 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-04-23T01:45:14.669 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_clone_cow": "true", 2024-04-23T01:45:14.669 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_compression_algorithm": "snappy", 2024-04-23T01:45:14.669 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_compression_max_blob_size": "0", 2024-04-23T01:45:14.670 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-04-23T01:45:14.670 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-04-23T01:45:14.670 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_compression_min_blob_size": "0", 2024-04-23T01:45:14.670 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-04-23T01:45:14.670 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-04-23T01:45:14.670 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_compression_mode": "none", 2024-04-23T01:45:14.670 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-04-23T01:45:14.670 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_csum_type": "crc32c", 2024-04-23T01:45:14.670 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_debug_enforce_min_alloc_size": "0", 2024-04-23T01:45:14.670 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_debug_enforce_settings": "default", 2024-04-23T01:45:14.670 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_debug_freelist": "false", 2024-04-23T01:45:14.670 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_debug_fsck_abort": "false", 2024-04-23T01:45:14.670 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-04-23T01:45:14.670 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-04-23T01:45:14.671 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_debug_inject_read_err": "false", 2024-04-23T01:45:14.671 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_debug_legacy_omap": "false", 2024-04-23T01:45:14.671 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-04-23T01:45:14.671 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-04-23T01:45:14.671 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-04-23T01:45:14.671 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-04-23T01:45:14.671 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_debug_prefill": "0.000000", 2024-04-23T01:45:14.671 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-04-23T01:45:14.671 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-04-23T01:45:14.671 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-04-23T01:45:14.671 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_debug_small_allocations": "0", 2024-04-23T01:45:14.671 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-04-23T01:45:14.671 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_default_buffered_read": "true", 2024-04-23T01:45:14.671 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_default_buffered_write": "false", 2024-04-23T01:45:14.672 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_deferred_batch_ops": "0", 2024-04-23T01:45:14.672 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-04-23T01:45:14.672 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-04-23T01:45:14.672 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_elastic_shared_blobs": "true", 2024-04-23T01:45:14.672 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-04-23T01:45:14.672 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-04-23T01:45:14.672 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-04-23T01:45:14.672 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-04-23T01:45:14.672 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-04-23T01:45:14.672 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fail_eio": "false", 2024-04-23T01:45:14.672 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-04-23T01:45:14.672 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-04-23T01:45:14.672 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-04-23T01:45:14.672 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-04-23T01:45:14.673 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_on_mkfs": "true", 2024-04-23T01:45:14.673 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-04-23T01:45:14.673 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_on_mount": "false", 2024-04-23T01:45:14.673 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-04-23T01:45:14.673 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_on_umount": "false", 2024-04-23T01:45:14.673 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-04-23T01:45:14.673 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-04-23T01:45:14.673 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-04-23T01:45:14.673 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-04-23T01:45:14.673 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-04-23T01:45:14.673 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-04-23T01:45:14.673 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-04-23T01:45:14.673 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-04-23T01:45:14.673 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_ignore_data_csum": "false", 2024-04-23T01:45:14.674 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-04-23T01:45:14.674 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_kvbackend": "rocksdb", 2024-04-23T01:45:14.674 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-04-23T01:45:14.674 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-04-23T01:45:14.674 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_log_op_age": "5.000000", 2024-04-23T01:45:14.674 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_max_alloc_size": "0", 2024-04-23T01:45:14.674 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_max_blob_size": "0", 2024-04-23T01:45:14.674 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-04-23T01:45:14.674 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-04-23T01:45:14.674 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_max_defer_interval": "3.000000", 2024-04-23T01:45:14.674 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_max_deferred_txc": "32", 2024-04-23T01:45:14.674 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_min_alloc_size": "0", 2024-04-23T01:45:14.674 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-04-23T01:45:14.674 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-04-23T01:45:14.675 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_nid_prealloc": "1024", 2024-04-23T01:45:14.675 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_prefer_deferred_size": "0", 2024-04-23T01:45:14.675 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-04-23T01:45:14.675 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-04-23T01:45:14.675 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_qfsck_on_mount": "true", 2024-04-23T01:45:14.675 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_retry_disk_reads": "3", 2024-04-23T01:45:14.675 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_rocksdb_cf": "true", 2024-04-23T01:45:14.675 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-04-23T01:45:14.675 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-04-23T01:45:14.675 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_rocksdb_options_annex": "", 2024-04-23T01:45:14.675 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_spdk_coremask": "0x1", 2024-04-23T01:45:14.675 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_spdk_io_sleep": "5", 2024-04-23T01:45:14.676 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_spdk_max_io_completion": "0", 2024-04-23T01:45:14.676 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_spdk_mem": "512", 2024-04-23T01:45:14.676 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_sync_submit_transaction": "false", 2024-04-23T01:45:14.676 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_throttle_bytes": "67108864", 2024-04-23T01:45:14.676 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_throttle_cost_per_io": "0", 2024-04-23T01:45:14.676 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-04-23T01:45:14.676 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-04-23T01:45:14.676 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-04-23T01:45:14.676 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-04-23T01:45:14.676 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_tracing": "false", 2024-04-23T01:45:14.676 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-04-23T01:45:14.676 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-04-23T01:45:14.676 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_volume_selection_reserved": "0", 2024-04-23T01:45:14.676 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-04-23T01:45:14.677 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-04-23T01:45:14.677 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-04-23T01:45:14.677 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-04-23T01:45:14.677 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-04-23T01:45:14.677 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-04-23T01:45:14.677 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_zero_block_detection": "false", 2024-04-23T01:45:14.677 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-04-23T01:45:14.677 INFO:teuthology.orchestra.run.smithi090.stdout: "cephfs_mirror_action_update_interval": "2", 2024-04-23T01:45:14.677 INFO:teuthology.orchestra.run.smithi090.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-04-23T01:45:14.677 INFO:teuthology.orchestra.run.smithi090.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-04-23T01:45:14.677 INFO:teuthology.orchestra.run.smithi090.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-04-23T01:45:14.677 INFO:teuthology.orchestra.run.smithi090.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-04-23T01:45:14.677 INFO:teuthology.orchestra.run.smithi090.stdout: "cephfs_mirror_mount_timeout": "10", 2024-04-23T01:45:14.677 INFO:teuthology.orchestra.run.smithi090.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-04-23T01:45:14.678 INFO:teuthology.orchestra.run.smithi090.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-04-23T01:45:14.678 INFO:teuthology.orchestra.run.smithi090.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-04-23T01:45:14.678 INFO:teuthology.orchestra.run.smithi090.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-04-23T01:45:14.678 INFO:teuthology.orchestra.run.smithi090.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-04-23T01:45:14.678 INFO:teuthology.orchestra.run.smithi090.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-04-23T01:45:14.678 INFO:teuthology.orchestra.run.smithi090.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-04-23T01:45:14.678 INFO:teuthology.orchestra.run.smithi090.stdout: "cephx_cluster_require_signatures": "false", 2024-04-23T01:45:14.678 INFO:teuthology.orchestra.run.smithi090.stdout: "cephx_cluster_require_version": "2", 2024-04-23T01:45:14.678 INFO:teuthology.orchestra.run.smithi090.stdout: "cephx_require_signatures": "false", 2024-04-23T01:45:14.678 INFO:teuthology.orchestra.run.smithi090.stdout: "cephx_require_version": "2", 2024-04-23T01:45:14.678 INFO:teuthology.orchestra.run.smithi090.stdout: "cephx_service_require_signatures": "false", 2024-04-23T01:45:14.678 INFO:teuthology.orchestra.run.smithi090.stdout: "cephx_service_require_version": "2", 2024-04-23T01:45:14.678 INFO:teuthology.orchestra.run.smithi090.stdout: "cephx_sign_messages": "true", 2024-04-23T01:45:14.679 INFO:teuthology.orchestra.run.smithi090.stdout: "chdir": "", 2024-04-23T01:45:14.679 INFO:teuthology.orchestra.run.smithi090.stdout: "client_acl_type": "", 2024-04-23T01:45:14.679 INFO:teuthology.orchestra.run.smithi090.stdout: "client_asio_thread_count": "2", 2024-04-23T01:45:14.679 INFO:teuthology.orchestra.run.smithi090.stdout: "client_cache_mid": "0.750000", 2024-04-23T01:45:14.679 INFO:teuthology.orchestra.run.smithi090.stdout: "client_cache_size": "16384", 2024-04-23T01:45:14.679 INFO:teuthology.orchestra.run.smithi090.stdout: "client_caps_release_delay": "5", 2024-04-23T01:45:14.679 INFO:teuthology.orchestra.run.smithi090.stdout: "client_check_pool_perm": "true", 2024-04-23T01:45:14.679 INFO:teuthology.orchestra.run.smithi090.stdout: "client_collect_and_send_global_metrics": "false", 2024-04-23T01:45:14.679 INFO:teuthology.orchestra.run.smithi090.stdout: "client_debug_force_sync_read": "false", 2024-04-23T01:45:14.679 INFO:teuthology.orchestra.run.smithi090.stdout: "client_debug_getattr_caps": "false", 2024-04-23T01:45:14.679 INFO:teuthology.orchestra.run.smithi090.stdout: "client_debug_inject_tick_delay": "0", 2024-04-23T01:45:14.679 INFO:teuthology.orchestra.run.smithi090.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-04-23T01:45:14.679 INFO:teuthology.orchestra.run.smithi090.stdout: "client_die_on_failed_remount": "false", 2024-04-23T01:45:14.679 INFO:teuthology.orchestra.run.smithi090.stdout: "client_dirsize_rbytes": "true", 2024-04-23T01:45:14.680 INFO:teuthology.orchestra.run.smithi090.stdout: "client_force_lazyio": "false", 2024-04-23T01:45:14.680 INFO:teuthology.orchestra.run.smithi090.stdout: "client_fs": "", 2024-04-23T01:45:14.680 INFO:teuthology.orchestra.run.smithi090.stdout: "client_inject_fixed_oldest_tid": "false", 2024-04-23T01:45:14.680 INFO:teuthology.orchestra.run.smithi090.stdout: "client_inject_release_failure": "false", 2024-04-23T01:45:14.680 INFO:teuthology.orchestra.run.smithi090.stdout: "client_max_inline_size": "4096", 2024-04-23T01:45:14.680 INFO:teuthology.orchestra.run.smithi090.stdout: "client_max_retries_on_remount_failure": "5", 2024-04-23T01:45:14.680 INFO:teuthology.orchestra.run.smithi090.stdout: "client_mds_namespace": "", 2024-04-23T01:45:14.680 INFO:teuthology.orchestra.run.smithi090.stdout: "client_metadata": "", 2024-04-23T01:45:14.680 INFO:teuthology.orchestra.run.smithi090.stdout: "client_mount_gid": "-1", 2024-04-23T01:45:14.680 INFO:teuthology.orchestra.run.smithi090.stdout: "client_mount_timeout": "300", 2024-04-23T01:45:14.680 INFO:teuthology.orchestra.run.smithi090.stdout: "client_mount_uid": "-1", 2024-04-23T01:45:14.680 INFO:teuthology.orchestra.run.smithi090.stdout: "client_mountpoint": "/", 2024-04-23T01:45:14.680 INFO:teuthology.orchestra.run.smithi090.stdout: "client_notify_timeout": "10", 2024-04-23T01:45:14.680 INFO:teuthology.orchestra.run.smithi090.stdout: "client_oc": "true", 2024-04-23T01:45:14.681 INFO:teuthology.orchestra.run.smithi090.stdout: "client_oc_max_dirty": "104857600", 2024-04-23T01:45:14.681 INFO:teuthology.orchestra.run.smithi090.stdout: "client_oc_max_dirty_age": "5.000000", 2024-04-23T01:45:14.681 INFO:teuthology.orchestra.run.smithi090.stdout: "client_oc_max_objects": "1000", 2024-04-23T01:45:14.681 INFO:teuthology.orchestra.run.smithi090.stdout: "client_oc_size": "209715200", 2024-04-23T01:45:14.681 INFO:teuthology.orchestra.run.smithi090.stdout: "client_oc_target_dirty": "8388608", 2024-04-23T01:45:14.681 INFO:teuthology.orchestra.run.smithi090.stdout: "client_permissions": "true", 2024-04-23T01:45:14.681 INFO:teuthology.orchestra.run.smithi090.stdout: "client_quota": "true", 2024-04-23T01:45:14.681 INFO:teuthology.orchestra.run.smithi090.stdout: "client_quota_df": "true", 2024-04-23T01:45:14.681 INFO:teuthology.orchestra.run.smithi090.stdout: "client_readahead_max_bytes": "0", 2024-04-23T01:45:14.681 INFO:teuthology.orchestra.run.smithi090.stdout: "client_readahead_max_periods": "4", 2024-04-23T01:45:14.681 INFO:teuthology.orchestra.run.smithi090.stdout: "client_readahead_min": "131072", 2024-04-23T01:45:14.681 INFO:teuthology.orchestra.run.smithi090.stdout: "client_reconnect_stale": "false", 2024-04-23T01:45:14.681 INFO:teuthology.orchestra.run.smithi090.stdout: "client_shutdown_timeout": "30", 2024-04-23T01:45:14.681 INFO:teuthology.orchestra.run.smithi090.stdout: "client_snapdir": ".snap", 2024-04-23T01:45:14.682 INFO:teuthology.orchestra.run.smithi090.stdout: "client_tick_interval": "1", 2024-04-23T01:45:14.682 INFO:teuthology.orchestra.run.smithi090.stdout: "client_trace": "", 2024-04-23T01:45:14.682 INFO:teuthology.orchestra.run.smithi090.stdout: "client_try_dentry_invalidate": "false", 2024-04-23T01:45:14.682 INFO:teuthology.orchestra.run.smithi090.stdout: "client_use_faked_inos": "false", 2024-04-23T01:45:14.682 INFO:teuthology.orchestra.run.smithi090.stdout: "client_use_random_mds": "false", 2024-04-23T01:45:14.682 INFO:teuthology.orchestra.run.smithi090.stdout: "clog_to_graylog": "false", 2024-04-23T01:45:14.682 INFO:teuthology.orchestra.run.smithi090.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-04-23T01:45:14.682 INFO:teuthology.orchestra.run.smithi090.stdout: "clog_to_graylog_port": "12201", 2024-04-23T01:45:14.682 INFO:teuthology.orchestra.run.smithi090.stdout: "clog_to_monitors": "default=true", 2024-04-23T01:45:14.682 INFO:teuthology.orchestra.run.smithi090.stdout: "clog_to_syslog": "false", 2024-04-23T01:45:14.682 INFO:teuthology.orchestra.run.smithi090.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-04-23T01:45:14.682 INFO:teuthology.orchestra.run.smithi090.stdout: "clog_to_syslog_level": "info", 2024-04-23T01:45:14.682 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_addr": "-", 2024-04-23T01:45:14.682 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_network": "", 2024-04-23T01:45:14.683 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_network_interface": "", 2024-04-23T01:45:14.683 INFO:teuthology.orchestra.run.smithi090.stdout: "compressor_zlib_isal": "false", 2024-04-23T01:45:14.683 INFO:teuthology.orchestra.run.smithi090.stdout: "compressor_zlib_level": "5", 2024-04-23T01:45:14.683 INFO:teuthology.orchestra.run.smithi090.stdout: "compressor_zlib_winsize": "-15", 2024-04-23T01:45:14.683 INFO:teuthology.orchestra.run.smithi090.stdout: "compressor_zstd_level": "1", 2024-04-23T01:45:14.683 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-04-23T01:45:14.683 INFO:teuthology.orchestra.run.smithi090.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-04-23T01:45:14.683 INFO:teuthology.orchestra.run.smithi090.stdout: "crimson_alien_op_num_threads": "6", 2024-04-23T01:45:14.683 INFO:teuthology.orchestra.run.smithi090.stdout: "crimson_alien_thread_cpu_cores": "", 2024-04-23T01:45:14.683 INFO:teuthology.orchestra.run.smithi090.stdout: "crimson_osd_obc_lru_size": "512", 2024-04-23T01:45:14.683 INFO:teuthology.orchestra.run.smithi090.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-04-23T01:45:14.683 INFO:teuthology.orchestra.run.smithi090.stdout: "crimson_osd_stat_interval": "0", 2024-04-23T01:45:14.683 INFO:teuthology.orchestra.run.smithi090.stdout: "crimson_seastar_cpu_cores": "", 2024-04-23T01:45:14.683 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_location": "", 2024-04-23T01:45:14.684 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_location_hook": "", 2024-04-23T01:45:14.684 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_location_hook_timeout": "10", 2024-04-23T01:45:14.684 INFO:teuthology.orchestra.run.smithi090.stdout: "daemonize": "false", 2024-04-23T01:45:14.684 INFO:teuthology.orchestra.run.smithi090.stdout: "daos_pool": "tank", 2024-04-23T01:45:14.684 INFO:teuthology.orchestra.run.smithi090.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-04-23T01:45:14.684 INFO:teuthology.orchestra.run.smithi090.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-04-23T01:45:14.684 INFO:teuthology.orchestra.run.smithi090.stdout: "dbstore_db_name_prefix": "dbstore", 2024-04-23T01:45:14.684 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_alienstore": "0/5", 2024-04-23T01:45:14.684 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_allow_any_pool_priority": "false", 2024-04-23T01:45:14.684 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_asok": "1/5", 2024-04-23T01:45:14.684 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_asok_assert_abort": "false", 2024-04-23T01:45:14.684 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_asserts_on_shutdown": "true", 2024-04-23T01:45:14.684 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_auth": "20/20", 2024-04-23T01:45:14.684 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_bdev": "1/3", 2024-04-23T01:45:14.685 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_bluefs": "1/5", 2024-04-23T01:45:14.685 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_bluestore": "1/5", 2024-04-23T01:45:14.685 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_buffer": "0/1", 2024-04-23T01:45:14.685 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_ceph_exporter": "1/5", 2024-04-23T01:45:14.685 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_cephfs_mirror": "0/5", 2024-04-23T01:45:14.685 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_cephsqlite": "0/5", 2024-04-23T01:45:14.685 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_client": "0/5", 2024-04-23T01:45:14.685 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_compressor": "1/5", 2024-04-23T01:45:14.685 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_context": "0/1", 2024-04-23T01:45:14.685 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_crimson_interrupt": "0/5", 2024-04-23T01:45:14.685 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_crush": "1/1", 2024-04-23T01:45:14.685 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_crypto": "1/5", 2024-04-23T01:45:14.685 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_cyanstore": "0/5", 2024-04-23T01:45:14.685 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_deliberately_leak_memory": "false", 2024-04-23T01:45:14.686 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_disable_randomized_ping": "false", 2024-04-23T01:45:14.686 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_dpdk": "1/5", 2024-04-23T01:45:14.686 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_eventtrace": "1/5", 2024-04-23T01:45:14.686 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_filer": "0/1", 2024-04-23T01:45:14.686 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_filestore": "1/3", 2024-04-23T01:45:14.686 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_finisher": "1/1", 2024-04-23T01:45:14.686 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_fuse": "1/5", 2024-04-23T01:45:14.686 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_heartbeat_testing_span": "0", 2024-04-23T01:45:14.686 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_heartbeatmap": "1/5", 2024-04-23T01:45:14.686 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_immutable_obj_cache": "0/5", 2024-04-23T01:45:14.686 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_javaclient": "1/5", 2024-04-23T01:45:14.686 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_journal": "1/3", 2024-04-23T01:45:14.686 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_journaler": "0/5", 2024-04-23T01:45:14.687 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_kstore": "1/5", 2024-04-23T01:45:14.687 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_lockdep": "0/1", 2024-04-23T01:45:14.687 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_mclock": "1/5", 2024-04-23T01:45:14.687 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_mds": "1/5", 2024-04-23T01:45:14.687 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_mds_balancer": "1/5", 2024-04-23T01:45:14.687 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_mds_locker": "1/5", 2024-04-23T01:45:14.687 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_mds_log": "1/5", 2024-04-23T01:45:14.687 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_mds_log_expire": "1/5", 2024-04-23T01:45:14.687 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_mds_migrator": "1/5", 2024-04-23T01:45:14.687 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_mds_quiesce": "3/5", 2024-04-23T01:45:14.687 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_memstore": "1/5", 2024-04-23T01:45:14.687 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_mgr": "2/5", 2024-04-23T01:45:14.687 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_mgrc": "1/5", 2024-04-23T01:45:14.687 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_mon": "20/20", 2024-04-23T01:45:14.687 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_monc": "0/10", 2024-04-23T01:45:14.688 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_ms": "1/1", 2024-04-23T01:45:14.688 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_none": "0/5", 2024-04-23T01:45:14.688 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_objclass": "0/5", 2024-04-23T01:45:14.688 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_objectcacher": "0/5", 2024-04-23T01:45:14.688 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_objecter": "0/1", 2024-04-23T01:45:14.688 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_optracker": "0/5", 2024-04-23T01:45:14.688 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_osd": "1/5", 2024-04-23T01:45:14.688 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_paxos": "20/20", 2024-04-23T01:45:14.688 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_perfcounter": "1/5", 2024-04-23T01:45:14.688 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_prioritycache": "1/5", 2024-04-23T01:45:14.688 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_rados": "0/5", 2024-04-23T01:45:14.688 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_rbd": "0/5", 2024-04-23T01:45:14.688 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_rbd_mirror": "0/5", 2024-04-23T01:45:14.688 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_rbd_pwl": "0/5", 2024-04-23T01:45:14.688 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_rbd_replay": "0/5", 2024-04-23T01:45:14.689 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_refs": "0/0", 2024-04-23T01:45:14.689 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_reserver": "1/1", 2024-04-23T01:45:14.689 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_rgw": "1/5", 2024-04-23T01:45:14.689 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_rgw_access": "1/5", 2024-04-23T01:45:14.689 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_rgw_datacache": "1/5", 2024-04-23T01:45:14.689 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_rgw_dbstore": "1/5", 2024-04-23T01:45:14.689 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_rgw_flight": "1/5", 2024-04-23T01:45:14.689 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_rgw_lifecycle": "1/5", 2024-04-23T01:45:14.689 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_rgw_notification": "1/5", 2024-04-23T01:45:14.689 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_rgw_sync": "1/5", 2024-04-23T01:45:14.689 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_rocksdb": "4/5", 2024-04-23T01:45:14.689 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_seastore": "0/5", 2024-04-23T01:45:14.689 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_seastore_backref": "0/5", 2024-04-23T01:45:14.689 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_seastore_cache": "0/5", 2024-04-23T01:45:14.690 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_seastore_cleaner": "0/5", 2024-04-23T01:45:14.690 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_seastore_device": "0/5", 2024-04-23T01:45:14.690 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_seastore_epm": "0/5", 2024-04-23T01:45:14.690 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-04-23T01:45:14.690 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_seastore_journal": "0/5", 2024-04-23T01:45:14.690 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_seastore_lba": "0/5", 2024-04-23T01:45:14.690 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_seastore_odata": "0/5", 2024-04-23T01:45:14.690 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_seastore_omap": "0/5", 2024-04-23T01:45:14.690 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_seastore_onode": "0/5", 2024-04-23T01:45:14.690 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_seastore_t": "0/5", 2024-04-23T01:45:14.690 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_seastore_tm": "0/5", 2024-04-23T01:45:14.690 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_striper": "0/1", 2024-04-23T01:45:14.690 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_test": "0/5", 2024-04-23T01:45:14.690 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_throttle": "1/1", 2024-04-23T01:45:14.691 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_timer": "0/1", 2024-04-23T01:45:14.691 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_tp": "0/5", 2024-04-23T01:45:14.691 INFO:teuthology.orchestra.run.smithi090.stdout: "debug_trace": "1/5", 2024-04-23T01:45:14.691 INFO:teuthology.orchestra.run.smithi090.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-04-23T01:45:14.691 INFO:teuthology.orchestra.run.smithi090.stdout: "device_failure_prediction_mode": "none", 2024-04-23T01:45:14.691 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "crimson", 2024-04-23T01:45:14.691 INFO:teuthology.orchestra.run.smithi090.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-04-23T01:45:14.691 INFO:teuthology.orchestra.run.smithi090.stdout: "err_to_graylog": "false", 2024-04-23T01:45:14.691 INFO:teuthology.orchestra.run.smithi090.stdout: "err_to_journald": "false", 2024-04-23T01:45:14.691 INFO:teuthology.orchestra.run.smithi090.stdout: "err_to_stderr": "true", 2024-04-23T01:45:14.691 INFO:teuthology.orchestra.run.smithi090.stdout: "err_to_syslog": "false", 2024-04-23T01:45:14.691 INFO:teuthology.orchestra.run.smithi090.stdout: "event_tracing": "false", 2024-04-23T01:45:14.691 INFO:teuthology.orchestra.run.smithi090.stdout: "exporter_addr": "0.0.0.0", 2024-04-23T01:45:14.692 INFO:teuthology.orchestra.run.smithi090.stdout: "exporter_http_port": "9926", 2024-04-23T01:45:14.692 INFO:teuthology.orchestra.run.smithi090.stdout: "exporter_prio_limit": "5", 2024-04-23T01:45:14.692 INFO:teuthology.orchestra.run.smithi090.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-04-23T01:45:14.692 INFO:teuthology.orchestra.run.smithi090.stdout: "exporter_sort_metrics": "true", 2024-04-23T01:45:14.692 INFO:teuthology.orchestra.run.smithi090.stdout: "exporter_stats_period": "5", 2024-04-23T01:45:14.692 INFO:teuthology.orchestra.run.smithi090.stdout: "fake_statfs_for_testing": "0", 2024-04-23T01:45:14.692 INFO:teuthology.orchestra.run.smithi090.stdout: "fatal_signal_handlers": "true", 2024-04-23T01:45:14.692 INFO:teuthology.orchestra.run.smithi090.stdout: "filer_max_purge_ops": "10", 2024-04-23T01:45:14.692 INFO:teuthology.orchestra.run.smithi090.stdout: "filer_max_truncate_ops": "128", 2024-04-23T01:45:14.692 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_apply_finisher_threads": "1", 2024-04-23T01:45:14.692 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_blackhole": "false", 2024-04-23T01:45:14.692 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_btrfs_clone_range": "true", 2024-04-23T01:45:14.692 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_btrfs_snap": "true", 2024-04-23T01:45:14.692 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_caller_concurrency": "10", 2024-04-23T01:45:14.692 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_collect_device_partition_information": "true", 2024-04-23T01:45:14.693 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_commit_timeout": "600.000000", 2024-04-23T01:45:14.693 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_debug_inject_read_err": "false", 2024-04-23T01:45:14.693 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_debug_omap_check": "false", 2024-04-23T01:45:14.693 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_debug_verify_split": "false", 2024-04-23T01:45:14.693 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_dump_file": "", 2024-04-23T01:45:14.693 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-04-23T01:45:14.693 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-04-23T01:45:14.693 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_fadvise": "true", 2024-04-23T01:45:14.693 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_fail_eio": "true", 2024-04-23T01:45:14.693 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_fd_cache_shards": "16", 2024-04-23T01:45:14.693 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_fd_cache_size": "128", 2024-04-23T01:45:14.693 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_fiemap": "false", 2024-04-23T01:45:14.693 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_fiemap_threshold": "4096", 2024-04-23T01:45:14.693 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-04-23T01:45:14.694 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_index_retry_probability": "0.000000", 2024-04-23T01:45:14.694 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_inject_stall": "0", 2024-04-23T01:45:14.694 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_journal_parallel": "false", 2024-04-23T01:45:14.694 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_journal_trailing": "false", 2024-04-23T01:45:14.694 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_journal_writeahead": "false", 2024-04-23T01:45:14.694 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_kill_at": "0", 2024-04-23T01:45:14.694 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-04-23T01:45:14.694 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_max_inline_xattr_size": "0", 2024-04-23T01:45:14.694 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-04-23T01:45:14.694 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-04-23T01:45:14.694 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-04-23T01:45:14.694 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_max_inline_xattrs": "0", 2024-04-23T01:45:14.694 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-04-23T01:45:14.694 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_max_inline_xattrs_other": "2", 2024-04-23T01:45:14.695 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-04-23T01:45:14.695 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_max_sync_interval": "5.000000", 2024-04-23T01:45:14.695 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_max_xattr_value_size": "0", 2024-04-23T01:45:14.695 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-04-23T01:45:14.695 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-04-23T01:45:14.695 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-04-23T01:45:14.695 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_merge_threshold": "-10", 2024-04-23T01:45:14.695 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_min_sync_interval": "0.010000", 2024-04-23T01:45:14.695 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_odsync_write": "false", 2024-04-23T01:45:14.695 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_omap_backend": "rocksdb", 2024-04-23T01:45:14.695 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_omap_backend_path": "", 2024-04-23T01:45:14.695 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_omap_header_cache_size": "1024", 2024-04-23T01:45:14.695 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_ondisk_finisher_threads": "1", 2024-04-23T01:45:14.695 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-04-23T01:45:14.696 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_op_thread_timeout": "60", 2024-04-23T01:45:14.696 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_op_threads": "2", 2024-04-23T01:45:14.696 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_punch_hole": "false", 2024-04-23T01:45:14.696 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-04-23T01:45:14.696 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-04-23T01:45:14.696 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-04-23T01:45:14.696 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-04-23T01:45:14.696 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-04-23T01:45:14.696 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_queue_max_bytes": "104857600", 2024-04-23T01:45:14.696 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-04-23T01:45:14.696 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-04-23T01:45:14.696 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-04-23T01:45:14.696 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_queue_max_ops": "50", 2024-04-23T01:45:14.696 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-04-23T01:45:14.696 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_seek_data_hole": "false", 2024-04-23T01:45:14.697 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_sloppy_crc": "false", 2024-04-23T01:45:14.697 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-04-23T01:45:14.697 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_splice": "false", 2024-04-23T01:45:14.697 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_split_multiple": "2", 2024-04-23T01:45:14.697 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_split_rand_factor": "20", 2024-04-23T01:45:14.697 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_update_to": "1000", 2024-04-23T01:45:14.697 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-04-23T01:45:14.697 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-04-23T01:45:14.697 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-04-23T01:45:14.697 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-04-23T01:45:14.697 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-04-23T01:45:14.697 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-04-23T01:45:14.697 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_wbthrottle_enable": "true", 2024-04-23T01:45:14.697 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-04-23T01:45:14.698 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-04-23T01:45:14.698 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-04-23T01:45:14.698 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-04-23T01:45:14.698 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-04-23T01:45:14.698 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-04-23T01:45:14.698 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_xfs_extsize": "false", 2024-04-23T01:45:14.698 INFO:teuthology.orchestra.run.smithi090.stdout: "filestore_zfs_snap": "false", 2024-04-23T01:45:14.698 INFO:teuthology.orchestra.run.smithi090.stdout: "fio_dir": "/tmp/fio", 2024-04-23T01:45:14.698 INFO:teuthology.orchestra.run.smithi090.stdout: "fsid": "0a731bfd-0fc4-4c1b-8e4b-f6dfbc059bcc", 2024-04-23T01:45:14.698 INFO:teuthology.orchestra.run.smithi090.stdout: "fuse_allow_other": "true", 2024-04-23T01:45:14.698 INFO:teuthology.orchestra.run.smithi090.stdout: "fuse_atomic_o_trunc": "true", 2024-04-23T01:45:14.698 INFO:teuthology.orchestra.run.smithi090.stdout: "fuse_big_writes": "true", 2024-04-23T01:45:14.698 INFO:teuthology.orchestra.run.smithi090.stdout: "fuse_debug": "false", 2024-04-23T01:45:14.698 INFO:teuthology.orchestra.run.smithi090.stdout: "fuse_default_permissions": "false", 2024-04-23T01:45:14.699 INFO:teuthology.orchestra.run.smithi090.stdout: "fuse_disable_pagecache": "false", 2024-04-23T01:45:14.699 INFO:teuthology.orchestra.run.smithi090.stdout: "fuse_max_write": "0", 2024-04-23T01:45:14.699 INFO:teuthology.orchestra.run.smithi090.stdout: "fuse_multithreaded": "true", 2024-04-23T01:45:14.699 INFO:teuthology.orchestra.run.smithi090.stdout: "fuse_require_active_mds": "true", 2024-04-23T01:45:14.699 INFO:teuthology.orchestra.run.smithi090.stdout: "fuse_set_user_groups": "true", 2024-04-23T01:45:14.699 INFO:teuthology.orchestra.run.smithi090.stdout: "fuse_splice_move": "true", 2024-04-23T01:45:14.699 INFO:teuthology.orchestra.run.smithi090.stdout: "fuse_splice_read": "true", 2024-04-23T01:45:14.699 INFO:teuthology.orchestra.run.smithi090.stdout: "fuse_splice_write": "true", 2024-04-23T01:45:14.699 INFO:teuthology.orchestra.run.smithi090.stdout: "fuse_syncfs_on_mksnap": "true", 2024-04-23T01:45:14.699 INFO:teuthology.orchestra.run.smithi090.stdout: "fuse_use_invalidate_cb": "true", 2024-04-23T01:45:14.699 INFO:teuthology.orchestra.run.smithi090.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-04-23T01:45:14.699 INFO:teuthology.orchestra.run.smithi090.stdout: "gss_target_name": "ceph", 2024-04-23T01:45:14.699 INFO:teuthology.orchestra.run.smithi090.stdout: "heartbeat_file": "", 2024-04-23T01:45:14.699 INFO:teuthology.orchestra.run.smithi090.stdout: "heartbeat_inject_failure": "0", 2024-04-23T01:45:14.700 INFO:teuthology.orchestra.run.smithi090.stdout: "heartbeat_interval": "5", 2024-04-23T01:45:14.700 INFO:teuthology.orchestra.run.smithi090.stdout: "host": "smithi090", 2024-04-23T01:45:14.700 INFO:teuthology.orchestra.run.smithi090.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-04-23T01:45:14.700 INFO:teuthology.orchestra.run.smithi090.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-04-23T01:45:14.700 INFO:teuthology.orchestra.run.smithi090.stdout: "immutable_object_cache_max_size": "1073741824", 2024-04-23T01:45:14.700 INFO:teuthology.orchestra.run.smithi090.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-04-23T01:45:14.700 INFO:teuthology.orchestra.run.smithi090.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-04-23T01:45:14.700 INFO:teuthology.orchestra.run.smithi090.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-04-23T01:45:14.700 INFO:teuthology.orchestra.run.smithi090.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-04-23T01:45:14.700 INFO:teuthology.orchestra.run.smithi090.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-04-23T01:45:14.700 INFO:teuthology.orchestra.run.smithi090.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-04-23T01:45:14.700 INFO:teuthology.orchestra.run.smithi090.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-04-23T01:45:14.701 INFO:teuthology.orchestra.run.smithi090.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-04-23T01:45:14.701 INFO:teuthology.orchestra.run.smithi090.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-04-23T01:45:14.701 INFO:teuthology.orchestra.run.smithi090.stdout: "immutable_object_cache_watermark": "0.900000", 2024-04-23T01:45:14.701 INFO:teuthology.orchestra.run.smithi090.stdout: "inject_early_sigterm": "false", 2024-04-23T01:45:14.701 INFO:teuthology.orchestra.run.smithi090.stdout: "jaeger_agent_port": "6799", 2024-04-23T01:45:14.701 INFO:teuthology.orchestra.run.smithi090.stdout: "jaeger_tracing_enable": "false", 2024-04-23T01:45:14.701 INFO:teuthology.orchestra.run.smithi090.stdout: "journal_aio": "true", 2024-04-23T01:45:14.701 INFO:teuthology.orchestra.run.smithi090.stdout: "journal_align_min_size": "65536", 2024-04-23T01:45:14.701 INFO:teuthology.orchestra.run.smithi090.stdout: "journal_block_align": "true", 2024-04-23T01:45:14.701 INFO:teuthology.orchestra.run.smithi090.stdout: "journal_block_size": "4096", 2024-04-23T01:45:14.701 INFO:teuthology.orchestra.run.smithi090.stdout: "journal_dio": "true", 2024-04-23T01:45:14.701 INFO:teuthology.orchestra.run.smithi090.stdout: "journal_discard": "false", 2024-04-23T01:45:14.701 INFO:teuthology.orchestra.run.smithi090.stdout: "journal_force_aio": "false", 2024-04-23T01:45:14.701 INFO:teuthology.orchestra.run.smithi090.stdout: "journal_ignore_corruption": "false", 2024-04-23T01:45:14.702 INFO:teuthology.orchestra.run.smithi090.stdout: "journal_max_write_bytes": "10485760", 2024-04-23T01:45:14.702 INFO:teuthology.orchestra.run.smithi090.stdout: "journal_max_write_entries": "100", 2024-04-23T01:45:14.702 INFO:teuthology.orchestra.run.smithi090.stdout: "journal_replay_from": "0", 2024-04-23T01:45:14.702 INFO:teuthology.orchestra.run.smithi090.stdout: "journal_throttle_high_multiple": "0.000000", 2024-04-23T01:45:14.702 INFO:teuthology.orchestra.run.smithi090.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-04-23T01:45:14.702 INFO:teuthology.orchestra.run.smithi090.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-04-23T01:45:14.702 INFO:teuthology.orchestra.run.smithi090.stdout: "journal_throttle_max_multiple": "0.000000", 2024-04-23T01:45:14.702 INFO:teuthology.orchestra.run.smithi090.stdout: "journal_write_header_frequency": "0", 2024-04-23T01:45:14.702 INFO:teuthology.orchestra.run.smithi090.stdout: "journal_zero_on_create": "false", 2024-04-23T01:45:14.702 INFO:teuthology.orchestra.run.smithi090.stdout: "journaler_prefetch_periods": "10", 2024-04-23T01:45:14.702 INFO:teuthology.orchestra.run.smithi090.stdout: "journaler_prezero_periods": "5", 2024-04-23T01:45:14.702 INFO:teuthology.orchestra.run.smithi090.stdout: "journaler_write_head_interval": "15", 2024-04-23T01:45:14.702 INFO:teuthology.orchestra.run.smithi090.stdout: "key": "", 2024-04-23T01:45:14.702 INFO:teuthology.orchestra.run.smithi090.stdout: "keyfile": "", 2024-04-23T01:45:14.703 INFO:teuthology.orchestra.run.smithi090.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-04-23T01:45:14.703 INFO:teuthology.orchestra.run.smithi090.stdout: "kstore_backend": "rocksdb", 2024-04-23T01:45:14.703 INFO:teuthology.orchestra.run.smithi090.stdout: "kstore_default_stripe_size": "65536", 2024-04-23T01:45:14.703 INFO:teuthology.orchestra.run.smithi090.stdout: "kstore_fsck_on_mount": "false", 2024-04-23T01:45:14.703 INFO:teuthology.orchestra.run.smithi090.stdout: "kstore_fsck_on_mount_deep": "true", 2024-04-23T01:45:14.703 INFO:teuthology.orchestra.run.smithi090.stdout: "kstore_max_bytes": "67108864", 2024-04-23T01:45:14.703 INFO:teuthology.orchestra.run.smithi090.stdout: "kstore_max_ops": "512", 2024-04-23T01:45:14.703 INFO:teuthology.orchestra.run.smithi090.stdout: "kstore_nid_prealloc": "1024", 2024-04-23T01:45:14.703 INFO:teuthology.orchestra.run.smithi090.stdout: "kstore_onode_map_size": "1024", 2024-04-23T01:45:14.703 INFO:teuthology.orchestra.run.smithi090.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-04-23T01:45:14.703 INFO:teuthology.orchestra.run.smithi090.stdout: "kstore_sync_submit_transaction": "false", 2024-04-23T01:45:14.703 INFO:teuthology.orchestra.run.smithi090.stdout: "kstore_sync_transaction": "false", 2024-04-23T01:45:14.703 INFO:teuthology.orchestra.run.smithi090.stdout: "librados_thread_count": "2", 2024-04-23T01:45:14.703 INFO:teuthology.orchestra.run.smithi090.stdout: "lockdep": "false", 2024-04-23T01:45:14.703 INFO:teuthology.orchestra.run.smithi090.stdout: "lockdep_force_backtrace": "false", 2024-04-23T01:45:14.704 INFO:teuthology.orchestra.run.smithi090.stdout: "log_coarse_timestamps": "true", 2024-04-23T01:45:14.704 INFO:teuthology.orchestra.run.smithi090.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-04-23T01:45:14.704 INFO:teuthology.orchestra.run.smithi090.stdout: "log_flush_on_exit": "false", 2024-04-23T01:45:14.704 INFO:teuthology.orchestra.run.smithi090.stdout: "log_graylog_host": "127.0.0.1", 2024-04-23T01:45:14.704 INFO:teuthology.orchestra.run.smithi090.stdout: "log_graylog_port": "12201", 2024-04-23T01:45:14.704 INFO:teuthology.orchestra.run.smithi090.stdout: "log_max_new": "1000", 2024-04-23T01:45:14.704 INFO:teuthology.orchestra.run.smithi090.stdout: "log_max_recent": "10000", 2024-04-23T01:45:14.704 INFO:teuthology.orchestra.run.smithi090.stdout: "log_stderr_prefix": "", 2024-04-23T01:45:14.704 INFO:teuthology.orchestra.run.smithi090.stdout: "log_stop_at_utilization": "0.970000", 2024-04-23T01:45:14.704 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": "true", 2024-04-23T01:45:14.704 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_graylog": "false", 2024-04-23T01:45:14.704 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_journald": "false", 2024-04-23T01:45:14.704 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_stderr": "false", 2024-04-23T01:45:14.705 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_syslog": "false", 2024-04-23T01:45:14.705 INFO:teuthology.orchestra.run.smithi090.stdout: "max_rotating_auth_attempts": "10", 2024-04-23T01:45:14.705 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-04-23T01:45:14.705 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_action_on_write_error": "1", 2024-04-23T01:45:14.705 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_alternate_name_max": "8192", 2024-04-23T01:45:14.705 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_asio_thread_count": "2", 2024-04-23T01:45:14.705 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_bal_export_pin": "true", 2024-04-23T01:45:14.705 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_bal_fragment_dirs": "true", 2024-04-23T01:45:14.705 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-04-23T01:45:14.705 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_bal_fragment_interval": "5", 2024-04-23T01:45:14.705 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_bal_fragment_size_max": "100000", 2024-04-23T01:45:14.705 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_bal_idle_threshold": "0.000000", 2024-04-23T01:45:14.705 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_bal_interval": "10", 2024-04-23T01:45:14.705 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_bal_max": "-1", 2024-04-23T01:45:14.706 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_bal_max_until": "-1", 2024-04-23T01:45:14.706 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_bal_merge_size": "50", 2024-04-23T01:45:14.706 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_bal_midchunk": "0.300000", 2024-04-23T01:45:14.706 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_bal_min_rebalance": "0.100000", 2024-04-23T01:45:14.706 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_bal_min_start": "0.200000", 2024-04-23T01:45:14.706 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_bal_minchunk": "0.001000", 2024-04-23T01:45:14.706 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_bal_mode": "0", 2024-04-23T01:45:14.706 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_bal_need_max": "1.200000", 2024-04-23T01:45:14.706 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_bal_need_min": "0.800000", 2024-04-23T01:45:14.706 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_bal_overload_epochs": "2", 2024-04-23T01:45:14.706 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-04-23T01:45:14.706 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_bal_sample_interval": "3.000000", 2024-04-23T01:45:14.706 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_bal_split_bits": "3", 2024-04-23T01:45:14.706 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_bal_split_rd": "25000.000000", 2024-04-23T01:45:14.707 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_bal_split_size": "10000", 2024-04-23T01:45:14.707 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_bal_split_wr": "10000.000000", 2024-04-23T01:45:14.707 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_bal_target_decay": "10.000000", 2024-04-23T01:45:14.707 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-04-23T01:45:14.707 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_beacon_grace": "15.000000", 2024-04-23T01:45:14.707 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_beacon_interval": "4.000000", 2024-04-23T01:45:14.707 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_beacon_mon_down_grace": "60", 2024-04-23T01:45:14.707 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_cache_memory_limit": "4294967296", 2024-04-23T01:45:14.707 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_cache_mid": "0.700000", 2024-04-23T01:45:14.707 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-04-23T01:45:14.707 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_cache_quiesce_delay": "0", 2024-04-23T01:45:14.707 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_cache_quiesce_sleep": "200", 2024-04-23T01:45:14.707 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_cache_quiesce_splitauth": "true", 2024-04-23T01:45:14.707 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_cache_quiesce_threshold": "524288", 2024-04-23T01:45:14.708 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_cache_release_free_interval": "10", 2024-04-23T01:45:14.708 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_cache_reservation": "0.050000", 2024-04-23T01:45:14.708 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-04-23T01:45:14.708 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_cache_trim_interval": "1", 2024-04-23T01:45:14.708 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_cache_trim_threshold": "262144", 2024-04-23T01:45:14.708 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-04-23T01:45:14.708 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-04-23T01:45:14.708 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_client_delegate_inos_pct": "50", 2024-04-23T01:45:14.708 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_client_prealloc_inos": "1000", 2024-04-23T01:45:14.708 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-04-23T01:45:14.708 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_connect_bootstrapping": "false", 2024-04-23T01:45:14.708 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_damage_table_max_entries": "10000", 2024-04-23T01:45:14.708 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-04-23T01:45:14.708 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_debug_auth_pins": "false", 2024-04-23T01:45:14.708 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_debug_frag": "false", 2024-04-23T01:45:14.709 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_debug_scatterstat": "false", 2024-04-23T01:45:14.709 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_debug_subtrees": "false", 2024-04-23T01:45:14.709 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_decay_halflife": "5.000000", 2024-04-23T01:45:14.709 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_default_dir_hash": "2", 2024-04-23T01:45:14.709 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_defer_session_stale": "true", 2024-04-23T01:45:14.709 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_deny_all_reconnect": "false", 2024-04-23T01:45:14.709 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_dir_keys_per_op": "16384", 2024-04-23T01:45:14.709 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_dir_max_commit_size": "10", 2024-04-23T01:45:14.709 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_dir_max_entries": "0", 2024-04-23T01:45:14.709 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_dir_prefetch": "true", 2024-04-23T01:45:14.709 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_dirstat_min_interval": "1.000000", 2024-04-23T01:45:14.709 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_dump_cache_after_rejoin": "false", 2024-04-23T01:45:14.709 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_dump_cache_on_map": "false", 2024-04-23T01:45:14.709 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_dump_cache_threshold_file": "0", 2024-04-23T01:45:14.710 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-04-23T01:45:14.710 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_early_reply": "true", 2024-04-23T01:45:14.710 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_enable_op_tracker": "true", 2024-04-23T01:45:14.710 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_enforce_unique_name": "true", 2024-04-23T01:45:14.710 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_export_ephemeral_distributed": "true", 2024-04-23T01:45:14.710 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-04-23T01:45:14.710 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_export_ephemeral_random": "true", 2024-04-23T01:45:14.710 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-04-23T01:45:14.710 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-04-23T01:45:14.710 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_forward_all_requests_to_auth": "false", 2024-04-23T01:45:14.710 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-04-23T01:45:14.710 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-04-23T01:45:14.710 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-04-23T01:45:14.710 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-04-23T01:45:14.710 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_health_cache_threshold": "1.500000", 2024-04-23T01:45:14.711 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_health_summarize_threshold": "10", 2024-04-23T01:45:14.711 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_heartbeat_grace": "15.000000", 2024-04-23T01:45:14.711 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_heartbeat_reset_grace": "1000", 2024-04-23T01:45:14.711 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_inject_health_dummy": "false", 2024-04-23T01:45:14.711 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-04-23T01:45:14.711 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_inject_migrator_session_race": "false", 2024-04-23T01:45:14.711 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-04-23T01:45:14.711 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-04-23T01:45:14.711 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-04-23T01:45:14.711 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_join_fs": "", 2024-04-23T01:45:14.712 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_journal_format": "1", 2024-04-23T01:45:14.712 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_kill_create_at": "0", 2024-04-23T01:45:14.712 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_kill_export_at": "0", 2024-04-23T01:45:14.712 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_kill_import_at": "0", 2024-04-23T01:45:14.712 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_kill_journal_at": "0", 2024-04-23T01:45:14.712 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_kill_journal_expire_at": "0", 2024-04-23T01:45:14.712 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_kill_journal_replay_at": "0", 2024-04-23T01:45:14.712 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_kill_link_at": "0", 2024-04-23T01:45:14.712 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_kill_mdstable_at": "0", 2024-04-23T01:45:14.712 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_kill_openc_at": "0", 2024-04-23T01:45:14.712 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_kill_rename_at": "0", 2024-04-23T01:45:14.712 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_kill_shutdown_at": "0", 2024-04-23T01:45:14.712 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-04-23T01:45:14.712 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_log_event_large_threshold": "524288", 2024-04-23T01:45:14.713 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_log_events_per_segment": "1024", 2024-04-23T01:45:14.713 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_log_major_segment_event_ratio": "12", 2024-04-23T01:45:14.713 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_log_max_events": "-1", 2024-04-23T01:45:14.713 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_log_max_segments": "128", 2024-04-23T01:45:14.713 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_log_pause": "false", 2024-04-23T01:45:14.713 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_log_segment_size": "0", 2024-04-23T01:45:14.713 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_log_skip_corrupt_events": "false", 2024-04-23T01:45:14.713 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_log_skip_unbounded_events": "false", 2024-04-23T01:45:14.713 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-04-23T01:45:14.713 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_log_trim_threshold": "128", 2024-04-23T01:45:14.713 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-04-23T01:45:14.713 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_log_warn_factor": "2.000000", 2024-04-23T01:45:14.713 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_max_caps_per_client": "1048576", 2024-04-23T01:45:14.713 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_max_completed_flushes": "100000", 2024-04-23T01:45:14.714 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_max_completed_requests": "100000", 2024-04-23T01:45:14.714 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_max_export_size": "20971520", 2024-04-23T01:45:14.714 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_max_file_recover": "32", 2024-04-23T01:45:14.714 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_max_purge_files": "64", 2024-04-23T01:45:14.714 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_max_purge_ops": "8192", 2024-04-23T01:45:14.714 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-04-23T01:45:14.714 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-04-23T01:45:14.714 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_max_snaps_per_dir": "100", 2024-04-23T01:45:14.714 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_metrics_update_interval": "2", 2024-04-23T01:45:14.714 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_min_caps_per_client": "100", 2024-04-23T01:45:14.714 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_min_caps_working_set": "10000", 2024-04-23T01:45:14.714 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-04-23T01:45:14.714 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_numa_node": "-1", 2024-04-23T01:45:14.714 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-04-23T01:45:14.715 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_op_complaint_time": "30.000000", 2024-04-23T01:45:14.715 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_op_history_duration": "600", 2024-04-23T01:45:14.715 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_op_history_size": "20", 2024-04-23T01:45:14.715 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_op_history_slow_op_size": "20", 2024-04-23T01:45:14.715 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_op_history_slow_op_threshold": "10", 2024-04-23T01:45:14.715 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_op_log_threshold": "5", 2024-04-23T01:45:14.715 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_ping_grace": "15", 2024-04-23T01:45:14.715 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_ping_interval": "5", 2024-04-23T01:45:14.715 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-04-23T01:45:14.715 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-04-23T01:45:14.715 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_recall_max_caps": "30000", 2024-04-23T01:45:14.715 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-04-23T01:45:14.715 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_recall_max_decay_threshold": "131072", 2024-04-23T01:45:14.715 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-04-23T01:45:14.715 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_recall_warning_threshold": "262144", 2024-04-23T01:45:14.716 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_reconnect_timeout": "45.000000", 2024-04-23T01:45:14.716 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_replay_interval": "1.000000", 2024-04-23T01:45:14.716 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-04-23T01:45:14.716 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-04-23T01:45:14.716 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_root_ino_gid": "0", 2024-04-23T01:45:14.716 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_root_ino_uid": "0", 2024-04-23T01:45:14.716 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-04-23T01:45:14.716 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_session_blocklist_on_evict": "true", 2024-04-23T01:45:14.716 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_session_blocklist_on_timeout": "true", 2024-04-23T01:45:14.716 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-04-23T01:45:14.716 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-04-23T01:45:14.716 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-04-23T01:45:14.716 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-04-23T01:45:14.716 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-04-23T01:45:14.717 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_session_metadata_threshold": "16777216", 2024-04-23T01:45:14.717 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-04-23T01:45:14.717 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_shutdown_check": "0", 2024-04-23T01:45:14.717 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_skip_ino": "0", 2024-04-23T01:45:14.717 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_sleep_rank_change": "0.000000", 2024-04-23T01:45:14.717 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_snap_max_uid": "4294967294", 2024-04-23T01:45:14.717 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_snap_min_uid": "0", 2024-04-23T01:45:14.717 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_snap_rstat": "false", 2024-04-23T01:45:14.717 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_standby_replay_damaged": "false", 2024-04-23T01:45:14.717 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_symlink_recovery": "true", 2024-04-23T01:45:14.717 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_task_status_update_interval": "2.000000", 2024-04-23T01:45:14.717 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_thrash_exports": "0", 2024-04-23T01:45:14.717 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_thrash_fragments": "0", 2024-04-23T01:45:14.717 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_tick_interval": "5.000000", 2024-04-23T01:45:14.717 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_valgrind_exit": "false", 2024-04-23T01:45:14.718 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_verify_backtrace": "1", 2024-04-23T01:45:14.718 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_verify_scatter": "false", 2024-04-23T01:45:14.718 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_wipe_ino_prealloc": "false", 2024-04-23T01:45:14.718 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_wipe_sessions": "false", 2024-04-23T01:45:14.718 INFO:teuthology.orchestra.run.smithi090.stdout: "mempool_debug": "false", 2024-04-23T01:45:14.718 INFO:teuthology.orchestra.run.smithi090.stdout: "memstore_debug_omit_block_device_write": "false", 2024-04-23T01:45:14.718 INFO:teuthology.orchestra.run.smithi090.stdout: "memstore_device_bytes": "1073741824", 2024-04-23T01:45:14.718 INFO:teuthology.orchestra.run.smithi090.stdout: "memstore_page_set": "false", 2024-04-23T01:45:14.718 INFO:teuthology.orchestra.run.smithi090.stdout: "memstore_page_size": "65536", 2024-04-23T01:45:14.718 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_client_bytes": "134217728", 2024-04-23T01:45:14.718 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_client_messages": "512", 2024-04-23T01:45:14.718 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-04-23T01:45:14.718 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_connect_retry_interval": "1.000000", 2024-04-23T01:45:14.718 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-04-23T01:45:14.719 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-04-23T01:45:14.719 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_disabled_modules": "", 2024-04-23T01:45:14.719 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-04-23T01:45:14.719 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_max_pg_creating": "1024", 2024-04-23T01:45:14.719 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_max_pg_num_change": "128", 2024-04-23T01:45:14.719 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_mds_bytes": "134217728", 2024-04-23T01:45:14.719 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_mds_messages": "128", 2024-04-23T01:45:14.719 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-04-23T01:45:14.719 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_mon_bytes": "134217728", 2024-04-23T01:45:14.719 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_mon_messages": "128", 2024-04-23T01:45:14.719 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_osd_bytes": "536870912", 2024-04-23T01:45:14.719 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_osd_messages": "8192", 2024-04-23T01:45:14.719 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_pool": "true", 2024-04-23T01:45:14.720 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_service_beacon_grace": "60.000000", 2024-04-23T01:45:14.720 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_standby_modules": "true", 2024-04-23T01:45:14.720 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_stats_period": "5", 2024-04-23T01:45:14.720 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_stats_threshold": "5", 2024-04-23T01:45:14.720 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_tick_period": "2", 2024-04-23T01:45:14.720 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-04-23T01:45:14.720 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_accept_timeout_factor": "2.000000", 2024-04-23T01:45:14.720 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_allow_pool_delete": "true", 2024-04-23T01:45:14.720 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_allow_pool_size_one": "true", 2024-04-23T01:45:14.720 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_auth_validate_all_caps": "true", 2024-04-23T01:45:14.720 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-04-23T01:45:14.720 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-04-23T01:45:14.720 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_client_bytes": "104857600", 2024-04-23T01:45:14.720 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_client_directed_command_retry": "2", 2024-04-23T01:45:14.720 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_client_hunt_interval": "3.000000", 2024-04-23T01:45:14.721 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-04-23T01:45:14.721 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-04-23T01:45:14.721 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-04-23T01:45:14.721 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_client_hunt_parallel": "3", 2024-04-23T01:45:14.721 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_client_log_interval": "1.000000", 2024-04-23T01:45:14.721 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-04-23T01:45:14.721 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_client_ping_interval": "10.000000", 2024-04-23T01:45:14.721 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_client_ping_timeout": "30.000000", 2024-04-23T01:45:14.721 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_client_target_rank": "-1", 2024-04-23T01:45:14.721 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_clock_drift_allowed": "1.000000", 2024-04-23T01:45:14.721 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-04-23T01:45:14.721 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-04-23T01:45:14.721 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_cluster_log_level": "debug", 2024-04-23T01:45:14.721 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_cluster_log_to_file": "true", 2024-04-23T01:45:14.722 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_cluster_log_to_graylog": "false", 2024-04-23T01:45:14.722 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-04-23T01:45:14.722 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-04-23T01:45:14.722 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_cluster_log_to_journald": "false", 2024-04-23T01:45:14.722 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_cluster_log_to_stderr": "false", 2024-04-23T01:45:14.722 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-04-23T01:45:14.722 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-04-23T01:45:14.722 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_compact_on_bootstrap": "false", 2024-04-23T01:45:14.722 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_compact_on_start": "false", 2024-04-23T01:45:14.722 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_compact_on_trim": "true", 2024-04-23T01:45:14.722 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_con_tracker_persist_interval": "10", 2024-04-23T01:45:14.722 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_con_tracker_score_halflife": "43200", 2024-04-23T01:45:14.722 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_config_key_max_entry_size": "65536", 2024-04-23T01:45:14.722 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_cpu_threads": "4", 2024-04-23T01:45:14.723 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_crush_min_required_version": "hammer", 2024-04-23T01:45:14.723 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_daemon_bytes": "419430400", 2024-04-23T01:45:14.723 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-04-23T01:45:14.723 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_data_avail_crit": "5", 2024-04-23T01:45:14.723 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_data_avail_warn": "5", 2024-04-23T01:45:14.723 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_data_size_warn": "16106127360", 2024-04-23T01:45:14.723 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_debug_block_osdmap_trim": "false", 2024-04-23T01:45:14.723 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-04-23T01:45:14.723 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_debug_dump_json": "false", 2024-04-23T01:45:14.723 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-04-23T01:45:14.723 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_debug_dump_transactions": "false", 2024-04-23T01:45:14.723 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_debug_extra_checks": "false", 2024-04-23T01:45:14.723 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-04-23T01:45:14.723 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-04-23T01:45:14.724 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_debug_no_require_reef": "false", 2024-04-23T01:45:14.724 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_debug_no_require_squid": "false", 2024-04-23T01:45:14.724 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-04-23T01:45:14.724 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_delta_reset_interval": "10.000000", 2024-04-23T01:45:14.724 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_dns_srv_name": "ceph-mon", 2024-04-23T01:45:14.724 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_down_mkfs_grace": "120", 2024-04-23T01:45:14.724 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_down_uptime_grace": "60", 2024-04-23T01:45:14.724 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_election_default_strategy": "1", 2024-04-23T01:45:14.724 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_election_timeout": "5.000000", 2024-04-23T01:45:14.724 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-04-23T01:45:14.724 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_elector_ping_divisor": "2", 2024-04-23T01:45:14.724 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_elector_ping_timeout": "2.000000", 2024-04-23T01:45:14.724 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_enable_op_tracker": "true", 2024-04-23T01:45:14.724 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_fake_pool_delete": "false", 2024-04-23T01:45:14.725 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_force_quorum_join": "false", 2024-04-23T01:45:14.725 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_globalid_prealloc": "10000", 2024-04-23T01:45:14.725 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_health_detail_to_clog": "false", 2024-04-23T01:45:14.725 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_health_log_update_period": "5", 2024-04-23T01:45:14.725 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_health_max_detail": "50", 2024-04-23T01:45:14.725 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_health_to_clog": "true", 2024-04-23T01:45:14.725 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_health_to_clog_interval": "600", 2024-04-23T01:45:14.725 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-04-23T01:45:14.725 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_host": "172.21.15.90", 2024-04-23T01:45:14.725 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_host_override": "", 2024-04-23T01:45:14.725 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_initial_members": "", 2024-04-23T01:45:14.725 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-04-23T01:45:14.725 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-04-23T01:45:14.725 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-04-23T01:45:14.725 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-04-23T01:45:14.726 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_keyvaluedb": "rocksdb", 2024-04-23T01:45:14.726 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_lease": "5.000000", 2024-04-23T01:45:14.726 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-04-23T01:45:14.726 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-04-23T01:45:14.726 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_log_full_interval": "50", 2024-04-23T01:45:14.726 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_log_max": "10000", 2024-04-23T01:45:14.726 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_log_max_summary": "50", 2024-04-23T01:45:14.726 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_max_log_entries_per_event": "4096", 2024-04-23T01:45:14.726 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_max_log_epochs": "500", 2024-04-23T01:45:14.726 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_max_mdsmap_epochs": "500", 2024-04-23T01:45:14.726 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_max_mgrmap_epochs": "500", 2024-04-23T01:45:14.726 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_max_osd": "10000", 2024-04-23T01:45:14.726 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_max_pg_per_osd": "10000", 2024-04-23T01:45:14.726 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_max_pool_pg_num": "65536", 2024-04-23T01:45:14.727 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-04-23T01:45:14.727 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-04-23T01:45:14.727 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_mds_force_trim_to": "0", 2024-04-23T01:45:14.727 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_mds_skip_sanity": "false", 2024-04-23T01:45:14.727 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_memory_autotune": "true", 2024-04-23T01:45:14.727 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_memory_target": "2147483648", 2024-04-23T01:45:14.727 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_mgr_beacon_grace": "30", 2024-04-23T01:45:14.727 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-04-23T01:45:14.727 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_mgr_digest_period": "5", 2024-04-23T01:45:14.727 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_mgr_inactive_grace": "60", 2024-04-23T01:45:14.727 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_mgr_mkfs_grace": "240", 2024-04-23T01:45:14.727 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-04-23T01:45:14.727 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_min_osdmap_epochs": "50", 2024-04-23T01:45:14.727 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_op_complaint_time": "30", 2024-04-23T01:45:14.727 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_op_history_duration": "600", 2024-04-23T01:45:14.728 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_op_history_size": "20", 2024-04-23T01:45:14.728 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_op_history_slow_op_size": "20", 2024-04-23T01:45:14.728 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_op_history_slow_op_threshold": "10", 2024-04-23T01:45:14.728 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_op_log_threshold": "5", 2024-04-23T01:45:14.728 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-04-23T01:45:14.728 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-04-23T01:45:14.728 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-04-23T01:45:14.728 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_auto_mark_in": "false", 2024-04-23T01:45:14.728 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_auto_mark_new_in": "true", 2024-04-23T01:45:14.728 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-04-23T01:45:14.728 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-04-23T01:45:14.728 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_cache_size": "500", 2024-04-23T01:45:14.728 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_cache_size_min": "134217728", 2024-04-23T01:45:14.728 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_crush_smoke_test": "true", 2024-04-23T01:45:14.728 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_destroyed_out_interval": "600", 2024-04-23T01:45:14.728 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_down_out_interval": "600", 2024-04-23T01:45:14.729 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-04-23T01:45:14.729 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-04-23T01:45:14.729 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_force_trim_to": "0", 2024-04-23T01:45:14.729 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_full_ratio": "0.950000", 2024-04-23T01:45:14.729 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-04-23T01:45:14.729 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_laggy_halflife": "3600", 2024-04-23T01:45:14.729 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_laggy_max_interval": "300", 2024-04-23T01:45:14.729 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_laggy_weight": "0.300000", 2024-04-23T01:45:14.729 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-04-23T01:45:14.729 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_max_initial_pgs": "1024", 2024-04-23T01:45:14.729 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_min_down_reporters": "2", 2024-04-23T01:45:14.729 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-04-23T01:45:14.729 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-04-23T01:45:14.729 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-04-23T01:45:14.730 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_prime_pg_temp": "true", 2024-04-23T01:45:14.730 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-04-23T01:45:14.730 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-04-23T01:45:14.730 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_report_timeout": "900", 2024-04-23T01:45:14.730 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-04-23T01:45:14.730 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-04-23T01:45:14.730 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_warn_num_repaired": "10", 2024-04-23T01:45:14.730 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osd_warn_op_age": "32.000000", 2024-04-23T01:45:14.730 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-04-23T01:45:14.730 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osdmap_full_prune_interval": "2", 2024-04-23T01:45:14.730 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osdmap_full_prune_min": "15", 2024-04-23T01:45:14.730 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_osdmap_full_prune_txsize": "2", 2024-04-23T01:45:14.730 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-04-23T01:45:14.730 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_pg_stuck_threshold": "60", 2024-04-23T01:45:14.730 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-04-23T01:45:14.731 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_pg_warn_min_objects": "10000", 2024-04-23T01:45:14.731 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_pg_warn_min_per_osd": "0", 2024-04-23T01:45:14.731 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-04-23T01:45:14.731 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_pool_quota_crit_threshold": "0", 2024-04-23T01:45:14.731 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_pool_quota_warn_threshold": "0", 2024-04-23T01:45:14.731 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_probe_timeout": "2.000000", 2024-04-23T01:45:14.731 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_reweight_max_change": "0.050000", 2024-04-23T01:45:14.731 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_reweight_max_osds": "4", 2024-04-23T01:45:14.731 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-04-23T01:45:14.731 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-04-23T01:45:14.731 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-04-23T01:45:14.731 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-04-23T01:45:14.731 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-04-23T01:45:14.731 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_scrub_interval": "86400", 2024-04-23T01:45:14.731 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_scrub_max_keys": "100", 2024-04-23T01:45:14.732 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_scrub_timeout": "300", 2024-04-23T01:45:14.732 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_session_timeout": "300", 2024-04-23T01:45:14.732 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_smart_report_timeout": "5", 2024-04-23T01:45:14.732 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_stat_smooth_intervals": "6", 2024-04-23T01:45:14.732 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-04-23T01:45:14.732 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_stretch_pool_min_size": "2", 2024-04-23T01:45:14.732 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_stretch_pool_size": "4", 2024-04-23T01:45:14.732 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-04-23T01:45:14.732 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_subscribe_interval": "86400.000000", 2024-04-23T01:45:14.732 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_sync_debug": "false", 2024-04-23T01:45:14.732 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_sync_max_payload_keys": "2000", 2024-04-23T01:45:14.732 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_sync_max_payload_size": "1048576", 2024-04-23T01:45:14.732 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_sync_provider_kill_at": "0", 2024-04-23T01:45:14.732 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_sync_requester_kill_at": "0", 2024-04-23T01:45:14.732 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_sync_timeout": "60.000000", 2024-04-23T01:45:14.733 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_target_pg_per_osd": "100", 2024-04-23T01:45:14.733 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_tick_interval": "5", 2024-04-23T01:45:14.733 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_timecheck_interval": "300.000000", 2024-04-23T01:45:14.733 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-04-23T01:45:14.733 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_use_min_delay_socket": "false", 2024-04-23T01:45:14.733 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_warn_older_version_delay": "604800", 2024-04-23T01:45:14.733 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-04-23T01:45:14.733 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-04-23T01:45:14.733 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-04-23T01:45:14.733 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_warn_on_filestore_osds": "false", 2024-04-23T01:45:14.733 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-04-23T01:45:14.733 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-04-23T01:45:14.733 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-04-23T01:45:14.733 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_warn_on_misplaced": "false", 2024-04-23T01:45:14.734 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-04-23T01:45:14.734 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_warn_on_older_version": "true", 2024-04-23T01:45:14.734 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-04-23T01:45:14.734 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_warn_on_pool_no_app": "true", 2024-04-23T01:45:14.734 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-04-23T01:45:14.734 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-04-23T01:45:14.734 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-04-23T01:45:14.734 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-04-23T01:45:14.734 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_warn_on_too_few_osds": "false", 2024-04-23T01:45:14.734 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-04-23T01:45:14.734 INFO:teuthology.orchestra.run.smithi090.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-04-23T01:45:14.734 INFO:teuthology.orchestra.run.smithi090.stdout: "monmap": "", 2024-04-23T01:45:14.734 INFO:teuthology.orchestra.run.smithi090.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-04-23T01:45:14.734 INFO:teuthology.orchestra.run.smithi090.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-04-23T01:45:14.735 INFO:teuthology.orchestra.run.smithi090.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-04-23T01:45:14.735 INFO:teuthology.orchestra.run.smithi090.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-04-23T01:45:14.735 INFO:teuthology.orchestra.run.smithi090.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-04-23T01:45:14.735 INFO:teuthology.orchestra.run.smithi090.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-04-23T01:45:14.735 INFO:teuthology.orchestra.run.smithi090.stdout: "motr_tracing_enabled": "false", 2024-04-23T01:45:14.735 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_async_op_threads": "3", 2024-04-23T01:45:14.735 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_async_rdma_buffer_size": "131072", 2024-04-23T01:45:14.735 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_async_rdma_cm": "false", 2024-04-23T01:45:14.735 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_async_rdma_device_name": "", 2024-04-23T01:45:14.735 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_async_rdma_dscp": "96", 2024-04-23T01:45:14.735 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-04-23T01:45:14.735 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_async_rdma_gid_idx": "0", 2024-04-23T01:45:14.735 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_async_rdma_local_gid": "", 2024-04-23T01:45:14.735 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_async_rdma_polling_us": "1000", 2024-04-23T01:45:14.736 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_async_rdma_port_num": "1", 2024-04-23T01:45:14.736 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-04-23T01:45:14.736 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-04-23T01:45:14.736 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_async_rdma_roce_ver": "1", 2024-04-23T01:45:14.736 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_async_rdma_send_buffers": "1024", 2024-04-23T01:45:14.736 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_async_rdma_sl": "3", 2024-04-23T01:45:14.736 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_async_rdma_support_srq": "true", 2024-04-23T01:45:14.736 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_async_rdma_type": "ib", 2024-04-23T01:45:14.736 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_async_reap_threshold": "5", 2024-04-23T01:45:14.736 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_bind_before_connect": "false", 2024-04-23T01:45:14.736 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_bind_ipv4": "true", 2024-04-23T01:45:14.736 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_bind_ipv6": "false", 2024-04-23T01:45:14.736 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_bind_msgr1": "true", 2024-04-23T01:45:14.737 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_bind_msgr2": "true", 2024-04-23T01:45:14.737 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_bind_port_max": "7568", 2024-04-23T01:45:14.737 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_bind_port_min": "6800", 2024-04-23T01:45:14.737 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_bind_prefer_ipv4": "false", 2024-04-23T01:45:14.737 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_bind_retry_count": "3", 2024-04-23T01:45:14.737 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_bind_retry_delay": "5", 2024-04-23T01:45:14.737 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_blackhole_client": "false", 2024-04-23T01:45:14.737 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_blackhole_mds": "false", 2024-04-23T01:45:14.737 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_blackhole_mgr": "false", 2024-04-23T01:45:14.737 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_blackhole_mon": "false", 2024-04-23T01:45:14.737 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_blackhole_osd": "false", 2024-04-23T01:45:14.737 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_client_mode": "crc", 2024-04-23T01:45:14.737 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_cluster_mode": "crc", 2024-04-23T01:45:14.737 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_cluster_type": "", 2024-04-23T01:45:14.737 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_compress_secure": "false", 2024-04-23T01:45:14.738 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_connection_idle_timeout": "900", 2024-04-23T01:45:14.738 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_connection_ready_timeout": "10", 2024-04-23T01:45:14.738 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_crc_data": "true", 2024-04-23T01:45:14.738 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_crc_header": "true", 2024-04-23T01:45:14.738 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_die_on_bad_msg": "false", 2024-04-23T01:45:14.738 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_die_on_bug": "true", 2024-04-23T01:45:14.738 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_die_on_old_message": "true", 2024-04-23T01:45:14.738 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_die_on_skipped_message": "false", 2024-04-23T01:45:14.738 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_die_on_unhandled_msg": "false", 2024-04-23T01:45:14.738 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-04-23T01:45:14.738 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_dpdk_coremask": "0xF", 2024-04-23T01:45:14.738 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-04-23T01:45:14.738 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_dpdk_devs_allowlist": "", 2024-04-23T01:45:14.738 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_dpdk_enable_tso": "true", 2024-04-23T01:45:14.739 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-04-23T01:45:14.739 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-04-23T01:45:14.739 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_dpdk_hugepages": "", 2024-04-23T01:45:14.739 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_dpdk_hw_flow_control": "true", 2024-04-23T01:45:14.739 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-04-23T01:45:14.739 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_dpdk_lro": "true", 2024-04-23T01:45:14.739 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_dpdk_memory_channel": "4", 2024-04-23T01:45:14.739 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-04-23T01:45:14.739 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_dpdk_pmd": "", 2024-04-23T01:45:14.739 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_dpdk_port_id": "0", 2024-04-23T01:45:14.739 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-04-23T01:45:14.739 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_dump_corrupt_message_level": "1", 2024-04-23T01:45:14.739 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_dump_on_send": "false", 2024-04-23T01:45:14.739 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_initial_backoff": "0.200000", 2024-04-23T01:45:14.739 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_inject_delay_max": "1.000000", 2024-04-23T01:45:14.740 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_inject_delay_probability": "0.000000", 2024-04-23T01:45:14.740 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_inject_delay_type": "", 2024-04-23T01:45:14.740 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_inject_internal_delays": "0.000000", 2024-04-23T01:45:14.740 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_inject_network_congestion": "0", 2024-04-23T01:45:14.740 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_inject_socket_failures": "0", 2024-04-23T01:45:14.740 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_learn_addr_from_peer": "true", 2024-04-23T01:45:14.740 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_max_accept_failures": "4", 2024-04-23T01:45:14.740 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_max_backoff": "15.000000", 2024-04-23T01:45:14.740 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_mon_client_mode": "crc", 2024-04-23T01:45:14.740 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_mon_cluster_mode": "crc", 2024-04-23T01:45:14.740 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_mon_service_mode": "crc", 2024-04-23T01:45:14.740 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_osd_compress_min_size": "1024", 2024-04-23T01:45:14.740 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_osd_compress_mode": "none", 2024-04-23T01:45:14.740 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_osd_compression_algorithm": "snappy", 2024-04-23T01:45:14.740 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-04-23T01:45:14.741 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_pq_min_cost": "65536", 2024-04-23T01:45:14.741 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_public_type": "", 2024-04-23T01:45:14.741 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_service_mode": "crc", 2024-04-23T01:45:14.741 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_tcp_listen_backlog": "512", 2024-04-23T01:45:14.741 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_tcp_nodelay": "true", 2024-04-23T01:45:14.741 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-04-23T01:45:14.741 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_tcp_rcvbuf": "0", 2024-04-23T01:45:14.741 INFO:teuthology.orchestra.run.smithi090.stdout: "ms_type": "async+posix", 2024-04-23T01:45:14.741 INFO:teuthology.orchestra.run.smithi090.stdout: "no_config_file": "false", 2024-04-23T01:45:14.741 INFO:teuthology.orchestra.run.smithi090.stdout: "objecter_completion_locks_per_session": "32", 2024-04-23T01:45:14.741 INFO:teuthology.orchestra.run.smithi090.stdout: "objecter_debug_inject_relock_delay": "false", 2024-04-23T01:45:14.741 INFO:teuthology.orchestra.run.smithi090.stdout: "objecter_inflight_op_bytes": "104857600", 2024-04-23T01:45:14.741 INFO:teuthology.orchestra.run.smithi090.stdout: "objecter_inflight_ops": "1024", 2024-04-23T01:45:14.741 INFO:teuthology.orchestra.run.smithi090.stdout: "objecter_inject_no_watch_ping": "false", 2024-04-23T01:45:14.741 INFO:teuthology.orchestra.run.smithi090.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-04-23T01:45:14.742 INFO:teuthology.orchestra.run.smithi090.stdout: "objecter_tick_interval": "5.000000", 2024-04-23T01:45:14.742 INFO:teuthology.orchestra.run.smithi090.stdout: "objecter_timeout": "10.000000", 2024-04-23T01:45:14.742 INFO:teuthology.orchestra.run.smithi090.stdout: "objectstore_blackhole": "false", 2024-04-23T01:45:14.742 INFO:teuthology.orchestra.run.smithi090.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-04-23T01:45:14.742 INFO:teuthology.orchestra.run.smithi090.stdout: "openssl_engine_opts": "", 2024-04-23T01:45:14.742 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_agent_delay_time": "5.000000", 2024-04-23T01:45:14.742 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_agent_hist_halflife": "1000", 2024-04-23T01:45:14.742 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_agent_max_low_ops": "2", 2024-04-23T01:45:14.742 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_agent_max_ops": "4", 2024-04-23T01:45:14.742 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-04-23T01:45:14.742 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_agent_quantize_effort": "0.100000", 2024-04-23T01:45:14.742 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_agent_slop": "0.020000", 2024-04-23T01:45:14.742 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-04-23T01:45:14.742 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_allow_recovery_below_min_size": "true", 2024-04-23T01:45:14.743 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_asio_thread_count": "2", 2024-04-23T01:45:14.743 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_async_recovery_min_cost": "100", 2024-04-23T01:45:14.743 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_auto_mark_unfound_lost": "false", 2024-04-23T01:45:14.743 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_backfill_retry_interval": "30.000000", 2024-04-23T01:45:14.743 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_backfill_scan_max": "512", 2024-04-23T01:45:14.743 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_backfill_scan_min": "64", 2024-04-23T01:45:14.743 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_backoff_on_degraded": "false", 2024-04-23T01:45:14.743 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_backoff_on_peering": "false", 2024-04-23T01:45:14.743 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_backoff_on_unfound": "true", 2024-04-23T01:45:14.743 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_beacon_report_interval": "300", 2024-04-23T01:45:14.743 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_bench_duration": "30", 2024-04-23T01:45:14.743 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-04-23T01:45:14.743 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_bench_max_block_size": "67108864", 2024-04-23T01:45:14.743 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_bench_small_size_max_iops": "100", 2024-04-23T01:45:14.743 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_blkin_trace_all": "false", 2024-04-23T01:45:14.743 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_blocked_scrub_grace_period": "120", 2024-04-23T01:45:14.744 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-04-23T01:45:14.744 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-04-23T01:45:14.744 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-04-23T01:45:14.744 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_check_for_log_corruption": "false", 2024-04-23T01:45:14.744 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-04-23T01:45:14.744 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-04-23T01:45:14.744 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-04-23T01:45:14.744 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-04-23T01:45:14.744 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_class_update_on_start": "true", 2024-04-23T01:45:14.744 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_client_message_cap": "256", 2024-04-23T01:45:14.744 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_client_message_size_cap": "524288000", 2024-04-23T01:45:14.744 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_client_op_priority": "63", 2024-04-23T01:45:14.744 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_client_watch_timeout": "30", 2024-04-23T01:45:14.744 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_command_max_records": "256", 2024-04-23T01:45:14.744 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_compact_on_start": "false", 2024-04-23T01:45:14.745 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-04-23T01:45:14.745 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_crush_chooseleaf_type": "0", 2024-04-23T01:45:14.745 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_crush_initial_weight": "-1.000000", 2024-04-23T01:45:14.745 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_crush_update_on_start": "true", 2024-04-23T01:45:14.745 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_crush_update_weight_set": "true", 2024-04-23T01:45:14.745 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-04-23T01:45:14.745 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-04-23T01:45:14.745 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-04-23T01:45:14.745 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_debug_drop_ping_duration": "0", 2024-04-23T01:45:14.745 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-04-23T01:45:14.745 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_debug_feed_pullee": "-1", 2024-04-23T01:45:14.745 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-04-23T01:45:14.745 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-04-23T01:45:14.745 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-04-23T01:45:14.745 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_debug_misdirected_ops": "false", 2024-04-23T01:45:14.746 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_debug_no_acting_change": "false", 2024-04-23T01:45:14.746 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_debug_no_purge_strays": "false", 2024-04-23T01:45:14.746 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_debug_op_order": "false", 2024-04-23T01:45:14.746 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_debug_pg_log_writeout": "false", 2024-04-23T01:45:14.746 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_debug_pretend_recovery_active": "false", 2024-04-23T01:45:14.746 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-04-23T01:45:14.746 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-04-23T01:45:14.746 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_debug_shutdown": "false", 2024-04-23T01:45:14.746 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-04-23T01:45:14.746 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-04-23T01:45:14.746 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_debug_trim_objects": "false", 2024-04-23T01:45:14.746 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_debug_verify_cached_snaps": "false", 2024-04-23T01:45:14.746 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_debug_verify_missing_on_start": "false", 2024-04-23T01:45:14.747 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_debug_verify_snaps": "false", 2024-04-23T01:45:14.747 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-04-23T01:45:14.747 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-04-23T01:45:14.747 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_deep_scrub_keys": "1024", 2024-04-23T01:45:14.747 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-04-23T01:45:14.747 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-04-23T01:45:14.747 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-04-23T01:45:14.747 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_deep_scrub_stride": "524288", 2024-04-23T01:45:14.747 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-04-23T01:45:14.747 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_default_data_pool_replay_window": "5", 2024-04-23T01:45:14.747 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_default_notify_timeout": "30", 2024-04-23T01:45:14.747 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_delete_sleep": "0.000000", 2024-04-23T01:45:14.747 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-04-23T01:45:14.747 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-04-23T01:45:14.748 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-04-23T01:45:14.748 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_discard_disconnected_ops": "true", 2024-04-23T01:45:14.748 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_enable_op_tracker": "true", 2024-04-23T01:45:14.748 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-04-23T01:45:14.748 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_extblkdev_plugins": "vdo", 2024-04-23T01:45:14.748 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-04-23T01:45:14.748 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-04-23T01:45:14.748 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_fast_info": "true", 2024-04-23T01:45:14.748 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_fast_shutdown": "true", 2024-04-23T01:45:14.748 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-04-23T01:45:14.748 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_fast_shutdown_timeout": "15", 2024-04-23T01:45:14.748 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-04-23T01:45:14.748 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-04-23T01:45:14.748 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-04-23T01:45:14.748 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_function_tracing": "false", 2024-04-23T01:45:14.748 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_heartbeat_grace": "20", 2024-04-23T01:45:14.749 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_heartbeat_interval": "6", 2024-04-23T01:45:14.749 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-04-23T01:45:14.749 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_heartbeat_min_peers": "10", 2024-04-23T01:45:14.749 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_heartbeat_min_size": "2000", 2024-04-23T01:45:14.749 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_heartbeat_stale": "600", 2024-04-23T01:45:14.749 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-04-23T01:45:14.749 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_hit_set_max_size": "100000", 2024-04-23T01:45:14.749 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_hit_set_min_size": "1000", 2024-04-23T01:45:14.749 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-04-23T01:45:14.749 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-04-23T01:45:14.749 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-04-23T01:45:14.749 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-04-23T01:45:14.749 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-04-23T01:45:14.749 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_journal_flush_on_shutdown": "true", 2024-04-23T01:45:14.750 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_journal_size": "5120", 2024-04-23T01:45:14.750 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_kill_backfill_at": "0", 2024-04-23T01:45:14.750 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_loop_before_reset_tphandle": "64", 2024-04-23T01:45:14.750 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_map_cache_size": "50", 2024-04-23T01:45:14.750 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_map_dedup": "true", 2024-04-23T01:45:14.750 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_map_message_max": "40", 2024-04-23T01:45:14.750 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_map_message_max_bytes": "10485760", 2024-04-23T01:45:14.750 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_map_share_max_epochs": "40", 2024-04-23T01:45:14.750 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_max_attr_name_len": "100", 2024-04-23T01:45:14.750 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_max_attr_size": "0", 2024-04-23T01:45:14.750 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_max_backfills": "1", 2024-04-23T01:45:14.750 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_max_markdown_count": "5", 2024-04-23T01:45:14.750 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_max_markdown_period": "600", 2024-04-23T01:45:14.750 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_max_object_name_len": "2048", 2024-04-23T01:45:14.751 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_max_object_namespace_len": "256", 2024-04-23T01:45:14.751 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_max_object_size": "134217728", 2024-04-23T01:45:14.751 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-04-23T01:45:14.751 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_max_omap_entries_per_request": "1024", 2024-04-23T01:45:14.751 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_max_pg_blocked_by": "16", 2024-04-23T01:45:14.751 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_max_pg_log_entries": "10000", 2024-04-23T01:45:14.751 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-04-23T01:45:14.751 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_max_pgls": "1024", 2024-04-23T01:45:14.751 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_max_push_cost": "8388608", 2024-04-23T01:45:14.751 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_max_push_objects": "10", 2024-04-23T01:45:14.751 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_max_scrubs": "3", 2024-04-23T01:45:14.751 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-04-23T01:45:14.751 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_max_trimming_pgs": "2", 2024-04-23T01:45:14.751 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_max_write_op_reply_len": "64", 2024-04-23T01:45:14.751 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_max_write_size": "90", 2024-04-23T01:45:14.752 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-04-23T01:45:14.752 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-04-23T01:45:14.752 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-04-23T01:45:14.752 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-04-23T01:45:14.752 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-04-23T01:45:14.752 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-04-23T01:45:14.752 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-04-23T01:45:14.752 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mclock_override_recovery_settings": "false", 2024-04-23T01:45:14.752 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mclock_profile": "balanced", 2024-04-23T01:45:14.752 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-04-23T01:45:14.752 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-04-23T01:45:14.752 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-04-23T01:45:14.752 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-04-23T01:45:14.752 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-04-23T01:45:14.753 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-04-23T01:45:14.753 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-04-23T01:45:14.753 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-04-23T01:45:14.753 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-04-23T01:45:14.753 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-04-23T01:45:14.753 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mclock_skip_benchmark": "false", 2024-04-23T01:45:14.753 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_memory_base": "805306368", 2024-04-23T01:45:14.753 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_memory_cache_min": "134217728", 2024-04-23T01:45:14.753 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-04-23T01:45:14.753 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-04-23T01:45:14.753 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_memory_target": "4294967296", 2024-04-23T01:45:14.753 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_memory_target_autotune": "false", 2024-04-23T01:45:14.753 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-04-23T01:45:14.753 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_min_pg_log_entries": "250", 2024-04-23T01:45:14.753 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_min_recovery_priority": "0", 2024-04-23T01:45:14.754 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mon_heartbeat_interval": "30", 2024-04-23T01:45:14.754 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-04-23T01:45:14.754 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mon_report_interval": "5", 2024-04-23T01:45:14.754 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mon_report_max_in_flight": "2", 2024-04-23T01:45:14.754 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-04-23T01:45:14.754 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_num_cache_shards": "32", 2024-04-23T01:45:14.754 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_num_op_tracker_shard": "32", 2024-04-23T01:45:14.754 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_numa_auto_affinity": "true", 2024-04-23T01:45:14.757 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_numa_node": "-1", 2024-04-23T01:45:14.758 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_numa_prefer_iface": "true", 2024-04-23T01:45:14.758 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-04-23T01:45:14.758 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_objecter_finishers": "1", 2024-04-23T01:45:14.758 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_objectstore": "bluestore", 2024-04-23T01:45:14.758 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_objectstore_fuse": "false", 2024-04-23T01:45:14.758 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_objectstore_tracing": "false", 2024-04-23T01:45:14.758 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_op_complaint_time": "30.000000", 2024-04-23T01:45:14.758 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_op_history_duration": "600", 2024-04-23T01:45:14.758 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_op_history_size": "20", 2024-04-23T01:45:14.758 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_op_history_slow_op_size": "20", 2024-04-23T01:45:14.758 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-04-23T01:45:14.758 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_op_log_threshold": "5", 2024-04-23T01:45:14.758 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_op_num_shards": "0", 2024-04-23T01:45:14.758 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_op_num_shards_hdd": "5", 2024-04-23T01:45:14.759 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_op_num_shards_ssd": "8", 2024-04-23T01:45:14.759 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_op_num_threads_per_shard": "0", 2024-04-23T01:45:14.759 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-04-23T01:45:14.759 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-04-23T01:45:14.759 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-04-23T01:45:14.759 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_op_pq_min_cost": "65536", 2024-04-23T01:45:14.759 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_op_queue": "mclock_scheduler", 2024-04-23T01:45:14.759 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_op_queue_cut_off": "high", 2024-04-23T01:45:14.759 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_op_thread_suicide_timeout": "150", 2024-04-23T01:45:14.759 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_op_thread_timeout": "15", 2024-04-23T01:45:14.759 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_open_classes_on_start": "true", 2024-04-23T01:45:14.759 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_os_flags": "0", 2024-04-23T01:45:14.759 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_peering_op_priority": "255", 2024-04-23T01:45:14.759 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pg_delete_cost": "1048576", 2024-04-23T01:45:14.759 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pg_delete_priority": "5", 2024-04-23T01:45:14.760 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-04-23T01:45:14.760 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-04-23T01:45:14.760 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pg_log_dups_tracked": "3000", 2024-04-23T01:45:14.760 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pg_log_trim_max": "10000", 2024-04-23T01:45:14.760 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pg_log_trim_min": "100", 2024-04-23T01:45:14.760 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-04-23T01:45:14.760 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pg_object_context_cache_count": "64", 2024-04-23T01:45:14.760 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-04-23T01:45:14.760 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-04-23T01:45:14.760 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-04-23T01:45:14.760 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-04-23T01:45:14.760 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-04-23T01:45:14.760 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-04-23T01:45:14.760 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-04-23T01:45:14.761 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-04-23T01:45:14.761 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pool_default_crimson": "true", 2024-04-23T01:45:14.761 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pool_default_crush_rule": "-1", 2024-04-23T01:45:14.761 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pool_default_ec_fast_read": "false", 2024-04-23T01:45:14.761 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-04-23T01:45:14.761 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pool_default_flag_bulk": "false", 2024-04-23T01:45:14.761 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-04-23T01:45:14.761 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pool_default_flag_nodelete": "false", 2024-04-23T01:45:14.761 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-04-23T01:45:14.761 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-04-23T01:45:14.761 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pool_default_flags": "0", 2024-04-23T01:45:14.761 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-04-23T01:45:14.761 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pool_default_min_size": "2", 2024-04-23T01:45:14.761 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-04-23T01:45:14.761 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pool_default_pg_num": "32", 2024-04-23T01:45:14.762 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pool_default_pgp_num": "0", 2024-04-23T01:45:14.762 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-04-23T01:45:14.762 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pool_default_read_ratio": "70", 2024-04-23T01:45:14.762 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pool_default_size": "3", 2024-04-23T01:45:14.762 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pool_default_type": "replicated", 2024-04-23T01:45:14.762 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-04-23T01:45:14.762 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pool_use_gmt_hitset": "true", 2024-04-23T01:45:14.762 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_push_per_object_cost": "1000", 2024-04-23T01:45:14.762 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_read_ec_check_for_errors": "false", 2024-04-23T01:45:14.762 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_recover_clone_overlap": "true", 2024-04-23T01:45:14.762 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_recover_clone_overlap_limit": "10", 2024-04-23T01:45:14.762 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_recovery_cost": "20971520", 2024-04-23T01:45:14.762 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_recovery_delay_start": "0.000000", 2024-04-23T01:45:14.763 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_recovery_max_active": "0", 2024-04-23T01:45:14.763 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_recovery_max_active_hdd": "3", 2024-04-23T01:45:14.763 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_recovery_max_active_ssd": "10", 2024-04-23T01:45:14.763 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_recovery_max_chunk": "8388608", 2024-04-23T01:45:14.763 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-04-23T01:45:14.763 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_recovery_max_single_start": "1", 2024-04-23T01:45:14.763 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_recovery_op_priority": "3", 2024-04-23T01:45:14.763 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_recovery_op_warn_multiple": "16", 2024-04-23T01:45:14.763 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_recovery_priority": "5", 2024-04-23T01:45:14.763 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_recovery_retry_interval": "30.000000", 2024-04-23T01:45:14.763 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_recovery_sleep": "0.000000", 2024-04-23T01:45:14.763 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-04-23T01:45:14.763 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-04-23T01:45:14.763 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-04-23T01:45:14.763 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_repair_during_recovery": "false", 2024-04-23T01:45:14.764 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_requested_scrub_priority": "120", 2024-04-23T01:45:14.764 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-04-23T01:45:14.764 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_rollback_to_cluster_snap": "", 2024-04-23T01:45:14.764 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_scrub_auto_repair": "false", 2024-04-23T01:45:14.764 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-04-23T01:45:14.764 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-04-23T01:45:14.764 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_scrub_begin_hour": "0", 2024-04-23T01:45:14.764 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_scrub_begin_week_day": "0", 2024-04-23T01:45:14.764 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_scrub_chunk_max": "25", 2024-04-23T01:45:14.764 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_scrub_chunk_min": "5", 2024-04-23T01:45:14.764 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_scrub_cost": "52428800", 2024-04-23T01:45:14.764 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-04-23T01:45:14.764 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_scrub_during_recovery": "false", 2024-04-23T01:45:14.764 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_scrub_end_hour": "0", 2024-04-23T01:45:14.765 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_scrub_end_week_day": "0", 2024-04-23T01:45:14.765 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_scrub_event_cost": "4096", 2024-04-23T01:45:14.765 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-04-23T01:45:14.765 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-04-23T01:45:14.765 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_scrub_invalid_stats": "true", 2024-04-23T01:45:14.765 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_scrub_load_threshold": "0.500000", 2024-04-23T01:45:14.765 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_scrub_max_interval": "604800.000000", 2024-04-23T01:45:14.765 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_scrub_max_preemptions": "5", 2024-04-23T01:45:14.765 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_scrub_min_interval": "86400.000000", 2024-04-23T01:45:14.765 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_scrub_priority": "5", 2024-04-23T01:45:14.765 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_scrub_reservation_timeout": "300000", 2024-04-23T01:45:14.765 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_scrub_sleep": "0.000000", 2024-04-23T01:45:14.765 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-04-23T01:45:14.765 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-04-23T01:45:14.765 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_shutdown_pgref_assert": "false", 2024-04-23T01:45:14.766 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_skip_check_past_interval_bounds": "false", 2024-04-23T01:45:14.766 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_skip_data_digest": "false", 2024-04-23T01:45:14.766 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_smart_report_timeout": "5", 2024-04-23T01:45:14.766 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_snap_trim_cost": "1048576", 2024-04-23T01:45:14.766 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_snap_trim_priority": "5", 2024-04-23T01:45:14.766 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_snap_trim_sleep": "0.000000", 2024-04-23T01:45:14.766 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-04-23T01:45:14.766 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-04-23T01:45:14.766 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-04-23T01:45:14.766 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-04-23T01:45:14.766 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_stats_update_period_scrubbing": "15", 2024-04-23T01:45:14.766 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-04-23T01:45:14.766 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_target_transaction_size": "30", 2024-04-23T01:45:14.766 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-04-23T01:45:14.767 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-04-23T01:45:14.767 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-04-23T01:45:14.767 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-04-23T01:45:14.767 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-04-23T01:45:14.767 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-04-23T01:45:14.767 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-04-23T01:45:14.767 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_tier_default_cache_mode": "writeback", 2024-04-23T01:45:14.767 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-04-23T01:45:14.767 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-04-23T01:45:14.767 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_tracing": "false", 2024-04-23T01:45:14.767 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_use_stale_snap": "false", 2024-04-23T01:45:14.767 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-04-23T01:45:14.767 INFO:teuthology.orchestra.run.smithi090.stdout: "osdc_blkin_trace_all": "false", 2024-04-23T01:45:14.767 INFO:teuthology.orchestra.run.smithi090.stdout: "paxos_kill_at": "0", 2024-04-23T01:45:14.767 INFO:teuthology.orchestra.run.smithi090.stdout: "paxos_max_join_drift": "10", 2024-04-23T01:45:14.768 INFO:teuthology.orchestra.run.smithi090.stdout: "paxos_min": "500", 2024-04-23T01:45:14.768 INFO:teuthology.orchestra.run.smithi090.stdout: "paxos_min_wait": "0.050000", 2024-04-23T01:45:14.768 INFO:teuthology.orchestra.run.smithi090.stdout: "paxos_propose_interval": "1.000000", 2024-04-23T01:45:14.768 INFO:teuthology.orchestra.run.smithi090.stdout: "paxos_service_trim_max": "500", 2024-04-23T01:45:14.768 INFO:teuthology.orchestra.run.smithi090.stdout: "paxos_service_trim_max_multiplier": "20", 2024-04-23T01:45:14.768 INFO:teuthology.orchestra.run.smithi090.stdout: "paxos_service_trim_min": "10", 2024-04-23T01:45:14.768 INFO:teuthology.orchestra.run.smithi090.stdout: "paxos_stash_full_interval": "25", 2024-04-23T01:45:14.768 INFO:teuthology.orchestra.run.smithi090.stdout: "paxos_trim_max": "500", 2024-04-23T01:45:14.768 INFO:teuthology.orchestra.run.smithi090.stdout: "paxos_trim_min": "250", 2024-04-23T01:45:14.768 INFO:teuthology.orchestra.run.smithi090.stdout: "perf": "true", 2024-04-23T01:45:14.768 INFO:teuthology.orchestra.run.smithi090.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-04-23T01:45:14.768 INFO:teuthology.orchestra.run.smithi090.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-04-23T01:45:14.768 INFO:teuthology.orchestra.run.smithi090.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-04-23T01:45:14.768 INFO:teuthology.orchestra.run.smithi090.stdout: "public_addr": "-", 2024-04-23T01:45:14.768 INFO:teuthology.orchestra.run.smithi090.stdout: "public_addrv": "", 2024-04-23T01:45:14.769 INFO:teuthology.orchestra.run.smithi090.stdout: "public_bind_addr": "-", 2024-04-23T01:45:14.769 INFO:teuthology.orchestra.run.smithi090.stdout: "public_network": "", 2024-04-23T01:45:14.769 INFO:teuthology.orchestra.run.smithi090.stdout: "public_network_interface": "", 2024-04-23T01:45:14.769 INFO:teuthology.orchestra.run.smithi090.stdout: "qat_compressor_enabled": "false", 2024-04-23T01:45:14.769 INFO:teuthology.orchestra.run.smithi090.stdout: "qat_compressor_session_max_number": "256", 2024-04-23T01:45:14.769 INFO:teuthology.orchestra.run.smithi090.stdout: "rados_mon_op_timeout": "0", 2024-04-23T01:45:14.769 INFO:teuthology.orchestra.run.smithi090.stdout: "rados_osd_op_timeout": "0", 2024-04-23T01:45:14.769 INFO:teuthology.orchestra.run.smithi090.stdout: "rados_tracing": "false", 2024-04-23T01:45:14.769 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_atime_update_interval": "60", 2024-04-23T01:45:14.769 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-04-23T01:45:14.769 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_balance_parent_reads": "false", 2024-04-23T01:45:14.769 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_balance_snap_reads": "false", 2024-04-23T01:45:14.769 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_blkin_trace_all": "false", 2024-04-23T01:45:14.769 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_blocklist_expire_seconds": "0", 2024-04-23T01:45:14.769 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_blocklist_on_break_lock": "true", 2024-04-23T01:45:14.770 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_cache": "true", 2024-04-23T01:45:14.770 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_cache_block_writes_upfront": "false", 2024-04-23T01:45:14.770 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_cache_max_dirty": "25165824", 2024-04-23T01:45:14.770 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-04-23T01:45:14.770 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_cache_max_dirty_object": "0", 2024-04-23T01:45:14.770 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_cache_policy": "writearound", 2024-04-23T01:45:14.770 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_cache_size": "33554432", 2024-04-23T01:45:14.770 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_cache_target_dirty": "16777216", 2024-04-23T01:45:14.770 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-04-23T01:45:14.770 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_clone_copy_on_read": "false", 2024-04-23T01:45:14.770 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_compression_hint": "none", 2024-04-23T01:45:14.770 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_concurrent_management_ops": "10", 2024-04-23T01:45:14.770 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-04-23T01:45:14.770 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_default_clone_format": "auto", 2024-04-23T01:45:14.771 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_default_data_pool": "", 2024-04-23T01:45:14.771 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_default_features": "61", 2024-04-23T01:45:14.771 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_default_format": "2", 2024-04-23T01:45:14.771 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_default_map_options": "", 2024-04-23T01:45:14.771 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_default_order": "22", 2024-04-23T01:45:14.771 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_default_pool": "rbd", 2024-04-23T01:45:14.771 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-04-23T01:45:14.771 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_default_stripe_count": "0", 2024-04-23T01:45:14.771 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_default_stripe_unit": "0", 2024-04-23T01:45:14.771 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_disable_zero_copy_writes": "true", 2024-04-23T01:45:14.771 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_discard_granularity_bytes": "65536", 2024-04-23T01:45:14.771 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-04-23T01:45:14.771 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_enable_alloc_hint": "true", 2024-04-23T01:45:14.771 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-04-23T01:45:14.772 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_io_scheduler": "simple", 2024-04-23T01:45:14.772 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-04-23T01:45:14.772 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_journal_commit_age": "5.000000", 2024-04-23T01:45:14.772 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-04-23T01:45:14.772 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-04-23T01:45:14.772 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-04-23T01:45:14.772 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-04-23T01:45:14.772 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_journal_object_flush_interval": "0", 2024-04-23T01:45:14.772 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-04-23T01:45:14.772 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-04-23T01:45:14.772 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_journal_order": "24", 2024-04-23T01:45:14.772 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_journal_pool": "", 2024-04-23T01:45:14.772 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_journal_splay_width": "4", 2024-04-23T01:45:14.772 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_localize_parent_reads": "false", 2024-04-23T01:45:14.772 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_localize_snap_reads": "false", 2024-04-23T01:45:14.773 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-04-23T01:45:14.773 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-04-23T01:45:14.773 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-04-23T01:45:14.773 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-04-23T01:45:14.773 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-04-23T01:45:14.773 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-04-23T01:45:14.773 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_mirror_image_policy_type": "simple", 2024-04-23T01:45:14.773 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-04-23T01:45:14.773 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-04-23T01:45:14.773 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-04-23T01:45:14.773 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-04-23T01:45:14.773 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-04-23T01:45:14.773 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-04-23T01:45:14.773 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-04-23T01:45:14.773 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_mirror_memory_autotune": "true", 2024-04-23T01:45:14.774 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_mirror_memory_base": "805306368", 2024-04-23T01:45:14.774 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-04-23T01:45:14.774 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-04-23T01:45:14.774 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-04-23T01:45:14.774 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-04-23T01:45:14.774 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_mirror_memory_target": "4294967296", 2024-04-23T01:45:14.774 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-04-23T01:45:14.774 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-04-23T01:45:14.774 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-04-23T01:45:14.774 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_mirroring_delete_delay": "0", 2024-04-23T01:45:14.774 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-04-23T01:45:14.774 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_mirroring_replay_delay": "0", 2024-04-23T01:45:14.774 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-04-23T01:45:14.774 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-04-23T01:45:14.775 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_move_to_trash_on_remove": "false", 2024-04-23T01:45:14.775 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-04-23T01:45:14.775 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_mtime_update_interval": "60", 2024-04-23T01:45:14.775 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_non_blocking_aio": "true", 2024-04-23T01:45:14.775 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_op_thread_timeout": "60", 2024-04-23T01:45:14.775 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_op_threads": "1", 2024-04-23T01:45:14.775 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_parent_cache_enabled": "false", 2024-04-23T01:45:14.775 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_persistent_cache_mode": "disabled", 2024-04-23T01:45:14.775 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_persistent_cache_path": "/tmp", 2024-04-23T01:45:14.775 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_persistent_cache_size": "1073741824", 2024-04-23T01:45:14.775 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_plugins": "", 2024-04-23T01:45:14.775 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_qos_bps_burst": "0", 2024-04-23T01:45:14.775 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-04-23T01:45:14.775 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_qos_bps_limit": "0", 2024-04-23T01:45:14.776 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_qos_exclude_ops": "0", 2024-04-23T01:45:14.776 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_qos_iops_burst": "0", 2024-04-23T01:45:14.776 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-04-23T01:45:14.776 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_qos_iops_limit": "0", 2024-04-23T01:45:14.776 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_qos_read_bps_burst": "0", 2024-04-23T01:45:14.776 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-04-23T01:45:14.776 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_qos_read_bps_limit": "0", 2024-04-23T01:45:14.776 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_qos_read_iops_burst": "0", 2024-04-23T01:45:14.776 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-04-23T01:45:14.776 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_qos_read_iops_limit": "0", 2024-04-23T01:45:14.776 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_qos_schedule_tick_min": "50", 2024-04-23T01:45:14.776 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_qos_write_bps_burst": "0", 2024-04-23T01:45:14.776 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-04-23T01:45:14.776 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_qos_write_bps_limit": "0", 2024-04-23T01:45:14.777 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_qos_write_iops_burst": "0", 2024-04-23T01:45:14.777 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-04-23T01:45:14.777 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_qos_write_iops_limit": "0", 2024-04-23T01:45:14.777 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_quiesce_notification_attempts": "10", 2024-04-23T01:45:14.777 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_read_from_replica_policy": "default", 2024-04-23T01:45:14.777 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-04-23T01:45:14.777 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_readahead_max_bytes": "524288", 2024-04-23T01:45:14.777 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_readahead_trigger_requests": "10", 2024-04-23T01:45:14.777 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_request_timed_out_seconds": "30", 2024-04-23T01:45:14.777 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_skip_partial_discard": "true", 2024-04-23T01:45:14.777 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-04-23T01:45:14.777 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_tracing": "false", 2024-04-23T01:45:14.777 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_validate_names": "true", 2024-04-23T01:45:14.777 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_validate_pool": "true", 2024-04-23T01:45:14.777 INFO:teuthology.orchestra.run.smithi090.stdout: "restapi_base_url": "", 2024-04-23T01:45:14.778 INFO:teuthology.orchestra.run.smithi090.stdout: "restapi_log_level": "", 2024-04-23T01:45:14.778 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_acl_grants_max_num": "100", 2024-04-23T01:45:14.778 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_admin_entry": "admin", 2024-04-23T01:45:14.778 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-04-23T01:45:14.778 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_backend_store": "rados", 2024-04-23T01:45:14.778 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_barbican_url": "", 2024-04-23T01:45:14.778 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_beast_enable_async": "true", 2024-04-23T01:45:14.778 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_bucket_counters_cache": "false", 2024-04-23T01:45:14.778 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-04-23T01:45:14.778 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-04-23T01:45:14.778 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-04-23T01:45:14.778 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_bucket_index_max_aio": "128", 2024-04-23T01:45:14.778 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-04-23T01:45:14.778 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-04-23T01:45:14.779 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_bucket_quota_ttl": "600", 2024-04-23T01:45:14.779 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-04-23T01:45:14.779 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_cache_enabled": "true", 2024-04-23T01:45:14.779 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_cache_expiry_interval": "900", 2024-04-23T01:45:14.779 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_cache_lru_size": "10000", 2024-04-23T01:45:14.779 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_config_store": "rados", 2024-04-23T01:45:14.779 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_content_length_compat": "false", 2024-04-23T01:45:14.779 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_copy_obj_progress": "true", 2024-04-23T01:45:14.779 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-04-23T01:45:14.779 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_cors_rules_max_num": "100", 2024-04-23T01:45:14.779 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_cross_domain_policy": "", 2024-04-23T01:45:14.779 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_default_encryption_key": "", 2024-04-23T01:45:14.779 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_kmip_addr": "", 2024-04-23T01:45:14.780 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_kmip_ca_path": "", 2024-04-23T01:45:14.780 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_kmip_client_cert": "", 2024-04-23T01:45:14.780 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_kmip_client_key": "", 2024-04-23T01:45:14.780 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-04-23T01:45:14.780 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_kmip_password": "", 2024-04-23T01:45:14.780 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-04-23T01:45:14.780 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_kmip_username": "", 2024-04-23T01:45:14.780 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_require_ssl": "true", 2024-04-23T01:45:14.780 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-04-23T01:45:14.780 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-04-23T01:45:14.780 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-04-23T01:45:14.780 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-04-23T01:45:14.780 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-04-23T01:45:14.780 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-04-23T01:45:14.781 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-04-23T01:45:14.781 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-04-23T01:45:14.781 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-04-23T01:45:14.781 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-04-23T01:45:14.781 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-04-23T01:45:14.781 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-04-23T01:45:14.781 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-04-23T01:45:14.781 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-04-23T01:45:14.781 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_suppress_logs": "true", 2024-04-23T01:45:14.781 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_vault_addr": "", 2024-04-23T01:45:14.781 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_vault_auth": "token", 2024-04-23T01:45:14.781 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_vault_namespace": "", 2024-04-23T01:45:14.781 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_vault_prefix": "", 2024-04-23T01:45:14.781 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-04-23T01:45:14.781 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-04-23T01:45:14.782 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-04-23T01:45:14.782 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-04-23T01:45:14.782 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_vault_token_file": "", 2024-04-23T01:45:14.782 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-04-23T01:45:14.782 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_curl_buffersize": "524288", 2024-04-23T01:45:14.782 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_curl_low_speed_limit": "1024", 2024-04-23T01:45:14.782 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_curl_low_speed_time": "300", 2024-04-23T01:45:14.782 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_curl_tcp_keepalive": "0", 2024-04-23T01:45:14.782 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-04-23T01:45:14.782 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-04-23T01:45:14.782 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-04-23T01:45:14.782 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-04-23T01:45:14.782 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-04-23T01:45:14.782 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_d3n_l1_fadvise": "4", 2024-04-23T01:45:14.783 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-04-23T01:45:14.783 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-04-23T01:45:14.783 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-04-23T01:45:14.783 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_d4n_host": "127.0.0.1", 2024-04-23T01:45:14.783 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_d4n_l1_datacache_persistent_path": "/tmp/rgw_d4n_datacache/", 2024-04-23T01:45:14.783 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_d4n_l1_datacache_size": "1073741824", 2024-04-23T01:45:14.783 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_d4n_l1_evict_cache_on_start": "true", 2024-04-23T01:45:14.783 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_d4n_l1_fadvise": "4", 2024-04-23T01:45:14.783 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_d4n_libaio_aio_num": "64", 2024-04-23T01:45:14.783 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_d4n_libaio_aio_threads": "20", 2024-04-23T01:45:14.783 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_d4n_port": "6379", 2024-04-23T01:45:14.783 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-04-23T01:45:14.783 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_data_log_changes_size": "1000", 2024-04-23T01:45:14.783 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_data_log_num_shards": "128", 2024-04-23T01:45:14.783 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-04-23T01:45:14.784 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_data_log_window": "30", 2024-04-23T01:45:14.784 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_data_notify_interval_msec": "0", 2024-04-23T01:45:14.784 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_data_sync_poll_interval": "20", 2024-04-23T01:45:14.784 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_data_sync_spawn_window": "20", 2024-04-23T01:45:14.784 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-04-23T01:45:14.784 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-04-23T01:45:14.784 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-04-23T01:45:14.784 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_default_data_log_backing": "fifo", 2024-04-23T01:45:14.784 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-04-23T01:45:14.784 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_default_region_info_oid": "default.region", 2024-04-23T01:45:14.784 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-04-23T01:45:14.784 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-04-23T01:45:14.784 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_defer_to_bucket_acls": "", 2024-04-23T01:45:14.784 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-04-23T01:45:14.784 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-04-23T01:45:14.785 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-04-23T01:45:14.785 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-04-23T01:45:14.785 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-04-23T01:45:14.785 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-04-23T01:45:14.785 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-04-23T01:45:14.785 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-04-23T01:45:14.785 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_dmclock_data_res": "500.000000", 2024-04-23T01:45:14.785 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-04-23T01:45:14.785 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-04-23T01:45:14.785 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-04-23T01:45:14.785 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-04-23T01:45:14.785 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_dns_name": "", 2024-04-23T01:45:14.785 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_dns_s3website_name": "", 2024-04-23T01:45:14.785 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_dynamic_resharding": "true", 2024-04-23T01:45:14.786 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-04-23T01:45:14.786 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_enable_gc_threads": "true", 2024-04-23T01:45:14.786 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_enable_lc_threads": "true", 2024-04-23T01:45:14.786 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_enable_ops_log": "false", 2024-04-23T01:45:14.786 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_enable_quota_threads": "true", 2024-04-23T01:45:14.786 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_enable_static_website": "false", 2024-04-23T01:45:14.786 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_enable_usage_log": "false", 2024-04-23T01:45:14.786 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_enforce_swift_acls": "true", 2024-04-23T01:45:14.786 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_exit_timeout_secs": "120", 2024-04-23T01:45:14.786 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_expose_bucket": "false", 2024-04-23T01:45:14.786 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_extended_http_attrs": "", 2024-04-23T01:45:14.786 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_filter": "none", 2024-04-23T01:45:14.786 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-04-23T01:45:14.786 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_frontends": "beast port=7480", 2024-04-23T01:45:14.786 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_gc_max_concurrent_io": "10", 2024-04-23T01:45:14.787 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_gc_max_deferred": "50", 2024-04-23T01:45:14.787 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-04-23T01:45:14.787 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_gc_max_objs": "32", 2024-04-23T01:45:14.787 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_gc_max_queue_size": "134213632", 2024-04-23T01:45:14.787 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_gc_max_trim_chunk": "16", 2024-04-23T01:45:14.787 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_gc_obj_min_wait": "7200", 2024-04-23T01:45:14.787 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_gc_processor_max_time": "3600", 2024-04-23T01:45:14.787 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_gc_processor_period": "3600", 2024-04-23T01:45:14.787 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-04-23T01:45:14.787 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_get_obj_window_size": "16777216", 2024-04-23T01:45:14.787 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_healthcheck_disabling_path": "", 2024-04-23T01:45:14.787 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_ignore_get_invalid_range": "false", 2024-04-23T01:45:14.787 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_init_timeout": "300", 2024-04-23T01:45:14.787 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-04-23T01:45:14.787 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-04-23T01:45:14.788 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_kafka_connection_idle": "300", 2024-04-23T01:45:14.788 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_kafka_message_timeout": "5000", 2024-04-23T01:45:14.788 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_kafka_sleep_timeout": "10", 2024-04-23T01:45:14.788 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-04-23T01:45:14.788 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-04-23T01:45:14.788 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-04-23T01:45:14.788 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_keystone_admin_domain": "", 2024-04-23T01:45:14.788 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_keystone_admin_password": "", 2024-04-23T01:45:14.788 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_keystone_admin_password_path": "", 2024-04-23T01:45:14.788 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_keystone_admin_project": "", 2024-04-23T01:45:14.788 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_keystone_admin_tenant": "", 2024-04-23T01:45:14.788 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_keystone_admin_token": "", 2024-04-23T01:45:14.788 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_keystone_admin_token_path": "", 2024-04-23T01:45:14.788 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_keystone_admin_user": "", 2024-04-23T01:45:14.788 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_keystone_api_version": "2", 2024-04-23T01:45:14.789 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_keystone_barbican_domain": "", 2024-04-23T01:45:14.789 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_keystone_barbican_password": "", 2024-04-23T01:45:14.789 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_keystone_barbican_project": "", 2024-04-23T01:45:14.789 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_keystone_barbican_tenant": "", 2024-04-23T01:45:14.789 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_keystone_barbican_user": "", 2024-04-23T01:45:14.789 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-04-23T01:45:14.789 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_keystone_implicit_tenants": "false", 2024-04-23T01:45:14.789 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-04-23T01:45:14.789 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_keystone_service_token_enabled": "false", 2024-04-23T01:45:14.789 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_keystone_token_cache_size": "10000", 2024-04-23T01:45:14.789 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_keystone_url": "", 2024-04-23T01:45:14.789 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_keystone_verify_ssl": "true", 2024-04-23T01:45:14.789 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_lc_debug_interval": "-1", 2024-04-23T01:45:14.789 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_lc_lock_max_time": "90", 2024-04-23T01:45:14.790 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_lc_max_objs": "32", 2024-04-23T01:45:14.790 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_lc_max_rules": "1000", 2024-04-23T01:45:14.790 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_lc_max_worker": "3", 2024-04-23T01:45:14.790 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_lc_max_wp_worker": "3", 2024-04-23T01:45:14.790 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_lc_thread_delay": "0", 2024-04-23T01:45:14.790 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-04-23T01:45:14.790 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_ldap_dnattr": "uid", 2024-04-23T01:45:14.790 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-04-23T01:45:14.790 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_ldap_searchfilter": "", 2024-04-23T01:45:14.790 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-04-23T01:45:14.790 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_ldap_uri": "", 2024-04-23T01:45:14.790 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_lfuda_sync_frequency": "60", 2024-04-23T01:45:14.790 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-04-23T01:45:14.790 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-04-23T01:45:14.790 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-04-23T01:45:14.791 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_local_cache_address": "127.0.0.1:6379", 2024-04-23T01:45:14.791 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_log_http_headers": "", 2024-04-23T01:45:14.791 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_log_nonexistent_bucket": "false", 2024-04-23T01:45:14.791 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-04-23T01:45:14.791 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_log_object_name_utc": "false", 2024-04-23T01:45:14.791 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-04-23T01:45:14.791 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-04-23T01:45:14.791 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_max_attr_name_len": "0", 2024-04-23T01:45:14.791 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_max_attr_size": "0", 2024-04-23T01:45:14.791 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_max_attrs_num_in_req": "0", 2024-04-23T01:45:14.791 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_max_chunk_size": "4194304", 2024-04-23T01:45:14.791 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_max_concurrent_requests": "1024", 2024-04-23T01:45:14.791 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-04-23T01:45:14.791 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_max_dynamic_shards": "1999", 2024-04-23T01:45:14.792 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_max_listing_results": "1000", 2024-04-23T01:45:14.792 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_max_notify_retries": "10", 2024-04-23T01:45:14.792 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_max_objs_per_shard": "100000", 2024-04-23T01:45:14.792 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_max_put_param_size": "1048576", 2024-04-23T01:45:14.792 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_max_put_size": "5368709120", 2024-04-23T01:45:14.792 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_max_slo_entries": "1000", 2024-04-23T01:45:14.792 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_md_log_max_shards": "64", 2024-04-23T01:45:14.792 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_md_notify_interval_msec": "200", 2024-04-23T01:45:14.792 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_meta_sync_poll_interval": "20", 2024-04-23T01:45:14.792 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_meta_sync_spawn_window": "20", 2024-04-23T01:45:14.792 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-04-23T01:45:14.792 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_mp_lock_max_time": "600", 2024-04-23T01:45:14.792 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-04-23T01:45:14.792 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_multipart_min_part_size": "5242880", 2024-04-23T01:45:14.792 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-04-23T01:45:14.793 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-04-23T01:45:14.793 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_nfs_fhcache_size": "2017", 2024-04-23T01:45:14.793 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-04-23T01:45:14.793 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-04-23T01:45:14.793 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_nfs_lru_lanes": "5", 2024-04-23T01:45:14.793 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_nfs_max_gc": "300", 2024-04-23T01:45:14.793 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-04-23T01:45:14.793 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_nfs_run_gc_threads": "false", 2024-04-23T01:45:14.793 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_nfs_run_lc_threads": "false", 2024-04-23T01:45:14.793 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_nfs_run_quota_threads": "false", 2024-04-23T01:45:14.793 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_nfs_run_sync_thread": "false", 2024-04-23T01:45:14.793 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-04-23T01:45:14.793 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-04-23T01:45:14.793 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_num_async_rados_threads": "32", 2024-04-23T01:45:14.794 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_num_control_oids": "8", 2024-04-23T01:45:14.794 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_numa_node": "-1", 2024-04-23T01:45:14.794 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_obj_stripe_size": "4194304", 2024-04-23T01:45:14.794 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-04-23T01:45:14.794 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_objexp_chunk_size": "100", 2024-04-23T01:45:14.794 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_objexp_gc_interval": "600", 2024-04-23T01:45:14.794 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_objexp_hints_num_shards": "127", 2024-04-23T01:45:14.794 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-04-23T01:45:14.794 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-04-23T01:45:14.794 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_op_thread_timeout": "600", 2024-04-23T01:45:14.794 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_op_tracing": "false", 2024-04-23T01:45:14.794 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_opa_token": "", 2024-04-23T01:45:14.794 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_opa_url": "", 2024-04-23T01:45:14.794 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_opa_verify_ssl": "true", 2024-04-23T01:45:14.795 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-04-23T01:45:14.795 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-04-23T01:45:14.795 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_ops_log_rados": "false", 2024-04-23T01:45:14.795 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_ops_log_socket_path": "", 2024-04-23T01:45:14.795 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-04-23T01:45:14.795 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-04-23T01:45:14.795 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-04-23T01:45:14.795 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_period_push_interval": "2.000000", 2024-04-23T01:45:14.795 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_period_push_interval_max": "30.000000", 2024-04-23T01:45:14.795 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_period_root_pool": ".rgw.root", 2024-04-23T01:45:14.795 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-04-23T01:45:14.795 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-04-23T01:45:14.795 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_posix_cache_lanes": "3", 2024-04-23T01:45:14.795 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-04-23T01:45:14.796 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_posix_cache_max_buckets": "100", 2024-04-23T01:45:14.796 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_posix_cache_partitions": "3", 2024-04-23T01:45:14.796 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-04-23T01:45:14.796 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_print_continue": "true", 2024-04-23T01:45:14.796 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_print_prohibited_content_length": "false", 2024-04-23T01:45:14.796 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-04-23T01:45:14.796 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-04-23T01:45:14.796 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-04-23T01:45:14.796 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-04-23T01:45:14.796 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_rados_tracing": "false", 2024-04-23T01:45:14.796 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_realm": "", 2024-04-23T01:45:14.796 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_realm_id": "", 2024-04-23T01:45:14.796 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-04-23T01:45:14.796 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_region": "", 2024-04-23T01:45:14.796 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_region_root_pool": ".rgw.root", 2024-04-23T01:45:14.796 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_relaxed_region_enforcement": "false", 2024-04-23T01:45:14.797 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-04-23T01:45:14.797 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-04-23T01:45:14.797 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_request_uri": "", 2024-04-23T01:45:14.797 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_reshard_batch_size": "64", 2024-04-23T01:45:14.797 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-04-23T01:45:14.797 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_reshard_max_aio": "128", 2024-04-23T01:45:14.797 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_reshard_num_logs": "16", 2024-04-23T01:45:14.797 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_reshard_thread_interval": "600", 2024-04-23T01:45:14.797 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_resolve_cname": "false", 2024-04-23T01:45:14.797 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_rest_getusage_op_compat": "false", 2024-04-23T01:45:14.797 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_run_sync_thread": "true", 2024-04-23T01:45:14.797 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-04-23T01:45:14.797 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-04-23T01:45:14.797 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_s3_auth_use_keystone": "false", 2024-04-23T01:45:14.797 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_s3_auth_use_ldap": "false", 2024-04-23T01:45:14.798 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_s3_auth_use_rados": "true", 2024-04-23T01:45:14.798 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_s3_auth_use_sts": "false", 2024-04-23T01:45:14.798 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-04-23T01:45:14.798 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_s3_success_create_obj_status": "0", 2024-04-23T01:45:14.798 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-04-23T01:45:14.798 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_scheduler_type": "throttler", 2024-04-23T01:45:14.798 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_script_uri": "", 2024-04-23T01:45:14.798 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_service_provider_name": "", 2024-04-23T01:45:14.798 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-04-23T01:45:14.798 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_sts_client_id": "", 2024-04-23T01:45:14.798 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_sts_client_secret": "", 2024-04-23T01:45:14.798 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_sts_entry": "sts", 2024-04-23T01:45:14.798 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_sts_key": "sts", 2024-04-23T01:45:14.798 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_sts_max_session_duration": "43200", 2024-04-23T01:45:14.799 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_sts_min_session_duration": "900", 2024-04-23T01:45:14.799 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_sts_token_introspection_url": "", 2024-04-23T01:45:14.799 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_swift_account_in_url": "false", 2024-04-23T01:45:14.799 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_swift_auth_entry": "auth", 2024-04-23T01:45:14.799 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_swift_auth_url": "", 2024-04-23T01:45:14.799 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_swift_custom_header": "", 2024-04-23T01:45:14.799 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_swift_enforce_content_length": "false", 2024-04-23T01:45:14.799 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_swift_need_stats": "true", 2024-04-23T01:45:14.799 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_swift_tenant_name": "", 2024-04-23T01:45:14.799 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_swift_token_expiration": "86400", 2024-04-23T01:45:14.799 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_swift_url": "", 2024-04-23T01:45:14.799 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_swift_url_prefix": "swift", 2024-04-23T01:45:14.799 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_swift_versioning_enabled": "false", 2024-04-23T01:45:14.799 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-04-23T01:45:14.800 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-04-23T01:45:14.800 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_sync_lease_period": "120", 2024-04-23T01:45:14.800 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-04-23T01:45:14.800 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_sync_log_trim_interval": "1200", 2024-04-23T01:45:14.800 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-04-23T01:45:14.800 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-04-23T01:45:14.801 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-04-23T01:45:14.801 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_sync_obj_etag_verify": "false", 2024-04-23T01:45:14.801 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_sync_trace_history_size": "4096", 2024-04-23T01:45:14.801 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-04-23T01:45:14.801 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-04-23T01:45:14.801 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_thread_pool_size": "512", 2024-04-23T01:45:14.801 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_topic_persistency_max_retries": "0", 2024-04-23T01:45:14.801 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-04-23T01:45:14.801 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-04-23T01:45:14.801 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_topic_require_publish_policy": "false", 2024-04-23T01:45:14.801 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_torrent_comment": "", 2024-04-23T01:45:14.801 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_torrent_createby": "", 2024-04-23T01:45:14.801 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_torrent_encoding": "", 2024-04-23T01:45:14.802 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_torrent_flag": "false", 2024-04-23T01:45:14.802 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_torrent_max_size": "5368709120", 2024-04-23T01:45:14.802 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_torrent_origin": "", 2024-04-23T01:45:14.802 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_torrent_sha_unit": "524288", 2024-04-23T01:45:14.802 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_torrent_tracker": "", 2024-04-23T01:45:14.802 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_trust_forwarded_https": "false", 2024-04-23T01:45:14.802 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-04-23T01:45:14.802 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_usage_log_tick_interval": "30", 2024-04-23T01:45:14.802 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_usage_max_shards": "32", 2024-04-23T01:45:14.802 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_usage_max_user_shards": "1", 2024-04-23T01:45:14.802 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_use_opa_authz": "false", 2024-04-23T01:45:14.802 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_user_counters_cache": "false", 2024-04-23T01:45:14.802 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_user_counters_cache_size": "10000", 2024-04-23T01:45:14.803 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-04-23T01:45:14.803 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_user_default_quota_max_size": "-1", 2024-04-23T01:45:14.803 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_user_max_buckets": "1000", 2024-04-23T01:45:14.803 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_user_policies_max_num": "100", 2024-04-23T01:45:14.803 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-04-23T01:45:14.803 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-04-23T01:45:14.803 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_user_quota_sync_interval": "86400", 2024-04-23T01:45:14.803 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-04-23T01:45:14.803 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_user_unique_email": "true", 2024-04-23T01:45:14.803 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_verify_ssl": "true", 2024-04-23T01:45:14.803 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_website_routing_rules_max_num": "50", 2024-04-23T01:45:14.803 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_zone": "", 2024-04-23T01:45:14.803 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_zone_id": "", 2024-04-23T01:45:14.803 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-04-23T01:45:14.803 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_zonegroup": "", 2024-04-23T01:45:14.804 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_zonegroup_id": "", 2024-04-23T01:45:14.804 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-04-23T01:45:14.804 INFO:teuthology.orchestra.run.smithi090.stdout: "rgwlc_auto_session_clear": "true", 2024-04-23T01:45:14.804 INFO:teuthology.orchestra.run.smithi090.stdout: "rgwlc_skip_bucket_step": "false", 2024-04-23T01:45:14.804 INFO:teuthology.orchestra.run.smithi090.stdout: "rocksdb_block_size": "4096", 2024-04-23T01:45:14.804 INFO:teuthology.orchestra.run.smithi090.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-04-23T01:45:14.804 INFO:teuthology.orchestra.run.smithi090.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-04-23T01:45:14.804 INFO:teuthology.orchestra.run.smithi090.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-04-23T01:45:14.804 INFO:teuthology.orchestra.run.smithi090.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-04-23T01:45:14.804 INFO:teuthology.orchestra.run.smithi090.stdout: "rocksdb_cache_shard_bits": "4", 2024-04-23T01:45:14.804 INFO:teuthology.orchestra.run.smithi090.stdout: "rocksdb_cache_size": "536870912", 2024-04-23T01:45:14.804 INFO:teuthology.orchestra.run.smithi090.stdout: "rocksdb_cache_type": "binned_lru", 2024-04-23T01:45:14.804 INFO:teuthology.orchestra.run.smithi090.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-04-23T01:45:14.804 INFO:teuthology.orchestra.run.smithi090.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-04-23T01:45:14.805 INFO:teuthology.orchestra.run.smithi090.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-04-23T01:45:14.805 INFO:teuthology.orchestra.run.smithi090.stdout: "rocksdb_collect_compaction_stats": "false", 2024-04-23T01:45:14.805 INFO:teuthology.orchestra.run.smithi090.stdout: "rocksdb_collect_extended_stats": "false", 2024-04-23T01:45:14.805 INFO:teuthology.orchestra.run.smithi090.stdout: "rocksdb_collect_memory_stats": "false", 2024-04-23T01:45:14.805 INFO:teuthology.orchestra.run.smithi090.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-04-23T01:45:14.805 INFO:teuthology.orchestra.run.smithi090.stdout: "rocksdb_index_type": "binary_search", 2024-04-23T01:45:14.805 INFO:teuthology.orchestra.run.smithi090.stdout: "rocksdb_log_to_ceph_log": "true", 2024-04-23T01:45:14.805 INFO:teuthology.orchestra.run.smithi090.stdout: "rocksdb_metadata_block_size": "4096", 2024-04-23T01:45:14.805 INFO:teuthology.orchestra.run.smithi090.stdout: "rocksdb_partition_filters": "false", 2024-04-23T01:45:14.805 INFO:teuthology.orchestra.run.smithi090.stdout: "rocksdb_perf": "false", 2024-04-23T01:45:14.805 INFO:teuthology.orchestra.run.smithi090.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-04-23T01:45:14.805 INFO:teuthology.orchestra.run.smithi090.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-04-23T01:45:14.805 INFO:teuthology.orchestra.run.smithi090.stdout: "rotating_keys_renewal_timeout": "10", 2024-04-23T01:45:14.805 INFO:teuthology.orchestra.run.smithi090.stdout: "run_dir": "/var/run/ceph", 2024-04-23T01:45:14.805 INFO:teuthology.orchestra.run.smithi090.stdout: "seastore_block_create": "true", 2024-04-23T01:45:14.806 INFO:teuthology.orchestra.run.smithi090.stdout: "seastore_cache_lru_size": "67108864", 2024-04-23T01:45:14.806 INFO:teuthology.orchestra.run.smithi090.stdout: "seastore_cbjournal_size": "5368709120", 2024-04-23T01:45:14.806 INFO:teuthology.orchestra.run.smithi090.stdout: "seastore_data_delta_based_overwrite": "0", 2024-04-23T01:45:14.806 INFO:teuthology.orchestra.run.smithi090.stdout: "seastore_default_max_object_size": "16777216", 2024-04-23T01:45:14.806 INFO:teuthology.orchestra.run.smithi090.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-04-23T01:45:14.806 INFO:teuthology.orchestra.run.smithi090.stdout: "seastore_device_size": "53687091200", 2024-04-23T01:45:14.806 INFO:teuthology.orchestra.run.smithi090.stdout: "seastore_journal_batch_capacity": "16", 2024-04-23T01:45:14.806 INFO:teuthology.orchestra.run.smithi090.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-04-23T01:45:14.806 INFO:teuthology.orchestra.run.smithi090.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-04-23T01:45:14.806 INFO:teuthology.orchestra.run.smithi090.stdout: "seastore_journal_iodepth_limit": "5", 2024-04-23T01:45:14.806 INFO:teuthology.orchestra.run.smithi090.stdout: "seastore_main_device_type": "SSD", 2024-04-23T01:45:14.806 INFO:teuthology.orchestra.run.smithi090.stdout: "seastore_max_concurrent_transactions": "8", 2024-04-23T01:45:14.806 INFO:teuthology.orchestra.run.smithi090.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-04-23T01:45:14.806 INFO:teuthology.orchestra.run.smithi090.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-04-23T01:45:14.807 INFO:teuthology.orchestra.run.smithi090.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-04-23T01:45:14.807 INFO:teuthology.orchestra.run.smithi090.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-04-23T01:45:14.807 INFO:teuthology.orchestra.run.smithi090.stdout: "seastore_segment_size": "67108864", 2024-04-23T01:45:14.807 INFO:teuthology.orchestra.run.smithi090.stdout: "set_keepcaps": "false", 2024-04-23T01:45:14.807 INFO:teuthology.orchestra.run.smithi090.stdout: "setgroup": "", 2024-04-23T01:45:14.807 INFO:teuthology.orchestra.run.smithi090.stdout: "setuser": "", 2024-04-23T01:45:14.807 INFO:teuthology.orchestra.run.smithi090.stdout: "setuser_match_path": "", 2024-04-23T01:45:14.807 INFO:teuthology.orchestra.run.smithi090.stdout: "target_max_misplaced_ratio": "0.050000", 2024-04-23T01:45:14.807 INFO:teuthology.orchestra.run.smithi090.stdout: "thp": "false", 2024-04-23T01:45:14.807 INFO:teuthology.orchestra.run.smithi090.stdout: "threadpool_default_timeout": "60", 2024-04-23T01:45:14.807 INFO:teuthology.orchestra.run.smithi090.stdout: "threadpool_empty_queue_max_wait": "2", 2024-04-23T01:45:14.807 INFO:teuthology.orchestra.run.smithi090.stdout: "throttler_perf_counter": "true" 2024-04-23T01:45:14.807 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-23T01:45:14.809 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-04-23T01:45:14.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:14.914+0000 7f9ef17ff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:14.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:14.920+0000 7f9ef17ff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:14.932 INFO:teuthology.orchestra.run.smithi090.stdout:{} 2024-04-23T01:45:14.932 INFO:teuthology.orchestra.run.smithi090.stderr:mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-23T01:45:14.943 DEBUG:teuthology.orchestra.run.smithi090:> type ceph-objectstore-tool 2024-04-23T01:45:14.957 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-23T01:45:14.957 DEBUG:teuthology.orchestra.run.smithi113:> type ceph-objectstore-tool 2024-04-23T01:45:14.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-23T01:45:14.975 INFO:teuthology.run_tasks:Running task full_sequential... 2024-04-23T01:45:14.984 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-04-23T01:45:14.986 INFO:tasks.radosbench:Beginning radosbench... 2024-04-23T01:45:14.986 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2024-04-23T01:45:14.986 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 2024-04-23T01:45:14.986 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-04-23T01:45:14.986 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:45:14.987 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:45:14.987 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:45:14.987 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-04-23T01:45:14.987 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:45:15.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:15.105+0000 7faf4d46d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:15.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:15.105+0000 7fd010aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:15.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:15.110+0000 7faf4d46d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:15.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:15.105+0000 7ffa1506d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:15.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:15.114+0000 7fd010aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:15.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:15.119+0000 7ffa1506d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:15.328 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:45:15.329 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:45:15.341 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-04-23T01:45:13.548705+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":107944,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377379416,"statfs":{"total":386547056640,"available":386436521984,"internally_reserved":0,"allocated":626688,"data_stored":1880100,"data_compressed":89859,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.969968"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.778515+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:45:10.778515+0000","last_peered":"2024-04-23T01:45:10.778515+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:45:10.778515+0000","last_undegraded":"2024-04-23T01:45:10.778515+0000","last_fullsized":"2024-04-23T01:45:10.778515+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.768599+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:45:10.768599+0000","last_peered":"2024-04-23T01:45:10.768599+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:45:10.768599+0000","last_undegraded":"2024-04-23T01:45:10.768599+0000","last_fullsized":"2024-04-23T01:45:10.768599+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T01:45:11.703427+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:45:11.703427+0000","last_peered":"2024-04-23T01:45:11.703427+0000","last_clean":"2024-04-23T01:45:11.703427+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:45:11.703427+0000","last_undegraded":"2024-04-23T01:45:11.703427+0000","last_fullsized":"2024-04-23T01:45:11.703427+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.765999+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:45:10.765999+0000","last_peered":"2024-04-23T01:45:10.765999+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:45:10.765999+0000","last_undegraded":"2024-04-23T01:45:10.765999+0000","last_fullsized":"2024-04-23T01:45:10.765999+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.782842+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:45:10.782842+0000","last_peered":"2024-04-23T01:45:10.782842+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:45:10.782842+0000","last_undegraded":"2024-04-23T01:45:10.782842+0000","last_fullsized":"2024-04-23T01:45:10.782842+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.753951+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:45:10.753951+0000","last_peered":"2024-04-23T01:45:10.753951+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:45:10.753951+0000","last_undegraded":"2024-04-23T01:45:10.753951+0000","last_fullsized":"2024-04-23T01:45:10.753951+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.770112+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:45:10.770112+0000","last_peered":"2024-04-23T01:45:10.770112+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:45:10.770112+0000","last_undegraded":"2024-04-23T01:45:10.770112+0000","last_fullsized":"2024-04-23T01:45:10.770112+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-23T01:45:10.803460+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:45:10.803460+0000","last_peered":"2024-04-23T01:45:10.803460+0000","last_clean":"2024-04-23T01:45:09.667353+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:45:10.803460+0000","last_undegraded":"2024-04-23T01:45:10.803460+0000","last_fullsized":"2024-04-23T01:45:10.803460+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":145,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T01:45:11.699272+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:45:11.699272+0000","last_peered":"2024-04-23T01:45:11.699272+0000","last_clean":"2024-04-23T01:45:11.699272+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:45:11.699272+0000","last_undegraded":"2024-04-23T01:45:11.699272+0000","last_fullsized":"2024-04-23T01:45:11.699272+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27004,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344836,"statfs":{"total":96636764160,"available":96609112064,"internally_reserved":0,"allocated":172032,"data_stored":617617,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27004,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344836,"statfs":{"total":96636764160,"available":96609112064,"internally_reserved":0,"allocated":172032,"data_stored":617617,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26932,"kb_used_data":108,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344908,"statfs":{"total":96636764160,"available":96609185792,"internally_reserved":0,"allocated":110592,"data_stored":27249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27004,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344836,"statfs":{"total":96636764160,"available":96609112064,"internally_reserved":0,"allocated":172032,"data_stored":617617,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:45:15.702 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'unique_pool_0' created 2024-04-23T01:45:15.719 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-04-23T01:45:15.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:15.839+0000 7fe85199d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:15.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:15.853+0000 7fe85199d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:16.705 INFO:teuthology.orchestra.run.smithi090.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-04-23T01:45:17.227 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:45:17.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:17.332+0000 7f929b66d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:17.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:17.337+0000 7f929b66d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:17.725 DEBUG:teuthology.orchestra.run.smithi090:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 8192 --object-size 65536 -p unique_pool_0 bench 90 write' 2024-04-23T01:45:17.728 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-04-23T01:45:17.728 INFO:teuthology.orchestra.run:waiting for 3000 2024-04-23T01:45:17.789 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:hints = 1 2024-04-23T01:45:17.789 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Maintaining 16 concurrent writes of 8192 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-04-23T01:45:17.789 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Object prefix: benchmark_data_smithi090_28592 2024-04-23T01:45:17.789 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:45:17.789 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 0 0 0 0 0 0 - 0 2024-04-23T01:45:18.789 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 1 16 174 158 1.2343 1.23438 0.0323873 0.0962216 2024-04-23T01:45:19.429 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:45:19.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:19.563+0000 7f6ba2f5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:19.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:19.570+0000 7f6ba2f5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:19.789 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 2 16 329 313 1.22257 1.21094 0.246572 0.0975304 2024-04-23T01:45:20.343 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:45:20.343 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:45:20.343 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:45:20.343 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:45:20.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:20.494+0000 7faaea66d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:20.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:20.504+0000 7faaea66d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:20.527 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:45:20.527 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:45:20.527 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:45:20.528 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:45:20.528 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:45:20.528 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:45:20.528 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:45:20.528 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:45:20.528 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:45:20.528 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:45:20.528 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:45:20.528 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:45:20.531 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:45:20.531 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:45:20.532 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:45:20.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:20.677+0000 7fd613f5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:20.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:20.689+0000 7fd613f5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:20.719 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:45:20.719 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:45:20.719 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:45:20.719 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:45:20.719 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:45:20.719 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:45:20.719 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:45:20.720 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:45:20.720 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:45:20.720 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:45:20.720 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:45:20.720 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:45:20.723 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:45:20.723 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:45:20.723 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:45:20.789 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 3 16 481 465 1.21084 1.1875 0.110316 0.101259 2024-04-23T01:45:20.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:20.841+0000 7fabc700f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:20.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:20.853+0000 7fabc700f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:20.874 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:45:20.874 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:45:20.874 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:45:20.874 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:45:20.874 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:45:20.874 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:45:20.874 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:45:20.874 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:45:20.874 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:45:20.874 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:45:20.874 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:45:20.874 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:45:20.877 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:45:20.877 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:45:20.877 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:45:21.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:21.007+0000 7fae6362d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:21.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:21.018+0000 7fae6362d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:21.050 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:45:21.050 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:45:21.050 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:45:21.050 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:45:21.050 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:45:21.051 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:45:21.051 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:45:21.051 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:45:21.051 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:45:21.051 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:45:21.051 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:45:21.051 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:45:21.053 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:45:21.054 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:45:21.742 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:45:21.789 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 4 16 646 630 1.23037 1.28906 0.0995019 0.10005 2024-04-23T01:45:21.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:21.873+0000 7f165d4ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:21.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:21.880+0000 7f165d4ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:22.790 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 5 16 829 813 1.27021 1.42969 0.0202501 0.0974816 2024-04-23T01:45:23.789 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 6 16 1002 986 1.28375 1.35156 0.0632928 0.0966829 2024-04-23T01:45:24.110 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:45:24.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:24.236+0000 7f7301aed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:24.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:24.242+0000 7f7301aed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:24.789 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 7 16 1178 1162 1.29677 1.375 0.0710221 0.0955553 2024-04-23T01:45:25.789 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 8 16 1340 1324 1.29286 1.26562 0.0218409 0.09599 2024-04-23T01:45:26.054 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:45:26.055 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:45:26.055 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:45:26.055 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:45:26.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:26.178+0000 7fe3fdead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:26.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:26.189+0000 7fe3fdead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:26.213 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:45:26.213 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:45:26.213 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:45:26.213 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:45:26.213 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:45:26.213 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:45:26.213 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:45:26.214 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:45:26.214 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:45:26.214 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:45:26.214 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:45:26.214 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:45:26.216 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:45:26.216 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:45:26.216 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:45:26.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:26.331+0000 7f241d46d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:26.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:26.340+0000 7f241d46d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:26.354 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:45:26.354 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:45:26.354 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:45:26.355 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:45:26.355 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:45:26.355 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:45:26.355 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:45:26.355 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:45:26.355 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:45:26.355 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:45:26.355 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:45:26.355 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:45:26.357 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:45:26.358 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:45:26.358 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:45:26.424 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:45:26.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:26.470+0000 7f591419d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:26.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:26.481+0000 7f591419d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:26.499 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:45:26.499 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:45:26.499 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:45:26.500 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:45:26.500 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:45:26.500 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:45:26.500 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:45:26.500 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:45:26.501 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:45:26.501 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:45:26.501 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:45:26.501 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:45:26.503 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:45:26.503 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:45:26.504 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:45:26.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:26.552+0000 7f9145406640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:26.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:26.559+0000 7f9145406640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:26.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:26.650+0000 7f2626e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:26.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:26.664+0000 7f2626e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:26.686 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:45:26.686 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:45:26.686 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:45:26.686 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:45:26.687 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:45:26.687 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:45:26.687 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:45:26.687 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:45:26.687 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:45:26.688 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:45:26.688 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:45:26.688 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:45:26.689 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:45:26.690 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:45:26.789 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 9 16 1472 1456 1.26379 1.03125 0.205814 0.0983688 2024-04-23T01:45:27.790 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 10 16 1652 1636 1.27802 1.40625 0.141367 0.0974257 2024-04-23T01:45:28.772 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:45:28.789 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 11 15 1808 1793 1.27333 1.22656 0.0919839 0.0976638 2024-04-23T01:45:28.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:28.905+0000 7f0ea4aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:28.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:28.911+0000 7f0ea4aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:29.790 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 12 16 1964 1948 1.26813 1.21094 0.0439907 0.0982417 2024-04-23T01:45:30.790 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 13 16 2127 2111 1.26853 1.27344 0.0979295 0.098165 2024-04-23T01:45:31.064 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:45:31.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:31.213+0000 7fd37839d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:31.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:31.221+0000 7fd37839d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:31.691 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:45:31.692 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-23T01:45:31.790 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 14 16 2278 2262 1.26217 1.17969 0.193249 0.0985502 2024-04-23T01:45:31.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:31.829+0000 7fc0532ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:31.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:31.841+0000 7fc0532ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:32.736 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.0 to 1 (10000) 2024-04-23T01:45:32.757 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-23T01:45:32.790 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 15 16 2445 2429 1.265 1.30469 0.0323038 0.0983201 2024-04-23T01:45:32.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:32.896+0000 7f0a5559d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:32.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:32.911+0000 7f0a5559d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:33.407 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:45:33.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:33.539+0000 7faef5e2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:33.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:33.546+0000 7faef5e2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:33.736 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.1 to 1 (10000) 2024-04-23T01:45:33.752 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-23T01:45:33.790 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 16 16 2591 2575 1.25722 1.14062 0.0356543 0.0989314 2024-04-23T01:45:33.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:33.921+0000 7f71ea20f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:33.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:33.933+0000 7f71ea20f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:34.743 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.2 to 1 (10000) 2024-04-23T01:45:34.763 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-23T01:45:34.790 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 17 16 2730 2714 1.24714 1.08594 0.123052 0.0996413 2024-04-23T01:45:34.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:34.896+0000 7f3cba66d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:34.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:34.909+0000 7f3cba66d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:35.731 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:45:35.749 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.3 to 1 (10000) 2024-04-23T01:45:35.765 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-23T01:45:35.765 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:45:35.790 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 18 16 2878 2862 1.24208 1.15625 0.0394062 0.100263 2024-04-23T01:45:35.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:35.849+0000 7f556cb5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:35.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:35.858+0000 7f556cb5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:35.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:35.926+0000 7f91056ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:35.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:35.938+0000 7f91056ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:36.202 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:45:36.203 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:45:36.218 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-04-23T01:45:35.751867+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":18899488,"num_objects":283,"num_object_clones":0,"num_object_copies":849,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":2372,"num_write_kb":19774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2316,"ondisk_log_size":2316,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":153788,"kb_used_data":25692,"kb_used_omap":0,"kb_used_meta":128064,"kb_avail":377333572,"statfs":{"total":386547056640,"available":386389577728,"internally_reserved":0,"allocated":26308608,"data_stored":53037684,"data_compressed":2725203,"data_compressed_allocated":25673728,"data_compressed_original":52822016,"omap_allocated":0,"internal_metadata":131137536},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6619136,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":808,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.199254"},"pg_stats":[{"pgid":"3.f","version":"17'119","reported_seq":132,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:31.134257+0000","last_change":"2024-04-23T01:45:16.856566+0000","last_active":"2024-04-23T01:45:31.134257+0000","last_peered":"2024-04-23T01:45:31.134257+0000","last_clean":"2024-04-23T01:45:31.134257+0000","last_became_active":"2024-04-23T01:45:16.848194+0000","last_became_peered":"2024-04-23T01:45:16.848194+0000","last_unstale":"2024-04-23T01:45:31.134257+0000","last_undegraded":"2024-04-23T01:45:31.134257+0000","last_fullsized":"2024-04-23T01:45:31.134257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"17'189","reported_seq":202,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:31.571329+0000","last_change":"2024-04-23T01:45:16.906007+0000","last_active":"2024-04-23T01:45:31.571329+0000","last_peered":"2024-04-23T01:45:31.571329+0000","last_clean":"2024-04-23T01:45:31.571329+0000","last_became_active":"2024-04-23T01:45:16.901215+0000","last_became_peered":"2024-04-23T01:45:16.901215+0000","last_unstale":"2024-04-23T01:45:31.571329+0000","last_undegraded":"2024-04-23T01:45:31.571329+0000","last_fullsized":"2024-04-23T01:45:31.571329+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1556480,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":190,"num_write_kb":1520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"17'151","reported_seq":164,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:31.185910+0000","last_change":"2024-04-23T01:45:16.886628+0000","last_active":"2024-04-23T01:45:31.185910+0000","last_peered":"2024-04-23T01:45:31.185910+0000","last_clean":"2024-04-23T01:45:31.185910+0000","last_became_active":"2024-04-23T01:45:16.880681+0000","last_became_peered":"2024-04-23T01:45:16.880681+0000","last_unstale":"2024-04-23T01:45:31.185910+0000","last_undegraded":"2024-04-23T01:45:31.185910+0000","last_fullsized":"2024-04-23T01:45:31.185910+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"17'103","reported_seq":116,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:30.446620+0000","last_change":"2024-04-23T01:45:16.836330+0000","last_active":"2024-04-23T01:45:30.446620+0000","last_peered":"2024-04-23T01:45:30.446620+0000","last_clean":"2024-04-23T01:45:30.446620+0000","last_became_active":"2024-04-23T01:45:16.821451+0000","last_became_peered":"2024-04-23T01:45:16.821451+0000","last_unstale":"2024-04-23T01:45:30.446620+0000","last_undegraded":"2024-04-23T01:45:30.446620+0000","last_fullsized":"2024-04-23T01:45:30.446620+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"17'103","reported_seq":116,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:28.512016+0000","last_change":"2024-04-23T01:45:16.860867+0000","last_active":"2024-04-23T01:45:28.512016+0000","last_peered":"2024-04-23T01:45:28.512016+0000","last_clean":"2024-04-23T01:45:28.512016+0000","last_became_active":"2024-04-23T01:45:16.852271+0000","last_became_peered":"2024-04-23T01:45:16.852271+0000","last_unstale":"2024-04-23T01:45:28.512016+0000","last_undegraded":"2024-04-23T01:45:28.512016+0000","last_fullsized":"2024-04-23T01:45:28.512016+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"17'167","reported_seq":180,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:29.681197+0000","last_change":"2024-04-23T01:45:16.805452+0000","last_active":"2024-04-23T01:45:29.681197+0000","last_peered":"2024-04-23T01:45:29.681197+0000","last_clean":"2024-04-23T01:45:29.681197+0000","last_became_active":"2024-04-23T01:45:16.797805+0000","last_became_peered":"2024-04-23T01:45:16.797805+0000","last_unstale":"2024-04-23T01:45:29.681197+0000","last_undegraded":"2024-04-23T01:45:29.681197+0000","last_fullsized":"2024-04-23T01:45:29.681197+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":1344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"17'143","reported_seq":156,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:30.853936+0000","last_change":"2024-04-23T01:45:16.893894+0000","last_active":"2024-04-23T01:45:30.853936+0000","last_peered":"2024-04-23T01:45:30.853936+0000","last_clean":"2024-04-23T01:45:30.853936+0000","last_became_active":"2024-04-23T01:45:16.888673+0000","last_became_peered":"2024-04-23T01:45:16.888673+0000","last_unstale":"2024-04-23T01:45:30.853936+0000","last_undegraded":"2024-04-23T01:45:30.853936+0000","last_fullsized":"2024-04-23T01:45:30.853936+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"17'127","reported_seq":140,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:31.372315+0000","last_change":"2024-04-23T01:45:16.889162+0000","last_active":"2024-04-23T01:45:31.372315+0000","last_peered":"2024-04-23T01:45:31.372315+0000","last_clean":"2024-04-23T01:45:31.372315+0000","last_became_active":"2024-04-23T01:45:16.880761+0000","last_became_peered":"2024-04-23T01:45:16.880761+0000","last_unstale":"2024-04-23T01:45:31.372315+0000","last_undegraded":"2024-04-23T01:45:31.372315+0000","last_fullsized":"2024-04-23T01:45:31.372315+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":154,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:18.010968+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:45:18.010968+0000","last_peered":"2024-04-23T01:45:18.010968+0000","last_clean":"2024-04-23T01:45:18.010968+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:45:18.010968+0000","last_undegraded":"2024-04-23T01:45:18.010968+0000","last_fullsized":"2024-04-23T01:45:18.010968+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:17.738806+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:45:17.738806+0000","last_peered":"2024-04-23T01:45:17.738806+0000","last_clean":"2024-04-23T01:45:17.738806+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:45:17.738806+0000","last_undegraded":"2024-04-23T01:45:17.738806+0000","last_fullsized":"2024-04-23T01:45:17.738806+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"17'127","reported_seq":140,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:31.270601+0000","last_change":"2024-04-23T01:45:16.891787+0000","last_active":"2024-04-23T01:45:31.270601+0000","last_peered":"2024-04-23T01:45:31.270601+0000","last_clean":"2024-04-23T01:45:31.270601+0000","last_became_active":"2024-04-23T01:45:16.885404+0000","last_became_peered":"2024-04-23T01:45:16.885404+0000","last_unstale":"2024-04-23T01:45:31.270601+0000","last_undegraded":"2024-04-23T01:45:31.270601+0000","last_fullsized":"2024-04-23T01:45:31.270601+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:17.848366+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:45:17.848366+0000","last_peered":"2024-04-23T01:45:17.848366+0000","last_clean":"2024-04-23T01:45:17.848366+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:45:17.848366+0000","last_undegraded":"2024-04-23T01:45:17.848366+0000","last_fullsized":"2024-04-23T01:45:17.848366+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"17'135","reported_seq":148,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:30.637767+0000","last_change":"2024-04-23T01:45:16.919259+0000","last_active":"2024-04-23T01:45:30.637767+0000","last_peered":"2024-04-23T01:45:30.637767+0000","last_clean":"2024-04-23T01:45:30.637767+0000","last_became_active":"2024-04-23T01:45:16.912983+0000","last_became_peered":"2024-04-23T01:45:16.912983+0000","last_unstale":"2024-04-23T01:45:30.637767+0000","last_undegraded":"2024-04-23T01:45:30.637767+0000","last_fullsized":"2024-04-23T01:45:30.637767+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":136,"num_write_kb":1088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:17.722899+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:45:17.722899+0000","last_peered":"2024-04-23T01:45:17.722899+0000","last_clean":"2024-04-23T01:45:17.722899+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:45:17.722899+0000","last_undegraded":"2024-04-23T01:45:17.722899+0000","last_fullsized":"2024-04-23T01:45:17.722899+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"17'104","reported_seq":117,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:31.275664+0000","last_change":"2024-04-23T01:45:16.897089+0000","last_active":"2024-04-23T01:45:31.275664+0000","last_peered":"2024-04-23T01:45:31.275664+0000","last_clean":"2024-04-23T01:45:31.275664+0000","last_became_active":"2024-04-23T01:45:16.891776+0000","last_became_peered":"2024-04-23T01:45:16.891776+0000","last_unstale":"2024-04-23T01:45:31.275664+0000","last_undegraded":"2024-04-23T01:45:31.275664+0000","last_fullsized":"2024-04-23T01:45:31.275664+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":860160,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105,"num_write_kb":840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:17.846928+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:45:17.846928+0000","last_peered":"2024-04-23T01:45:17.846928+0000","last_clean":"2024-04-23T01:45:17.846928+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:45:17.846928+0000","last_undegraded":"2024-04-23T01:45:17.846928+0000","last_fullsized":"2024-04-23T01:45:17.846928+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"17'103","reported_seq":116,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:29.787991+0000","last_change":"2024-04-23T01:45:16.823909+0000","last_active":"2024-04-23T01:45:29.787991+0000","last_peered":"2024-04-23T01:45:29.787991+0000","last_clean":"2024-04-23T01:45:29.787991+0000","last_became_active":"2024-04-23T01:45:16.815467+0000","last_became_peered":"2024-04-23T01:45:16.815467+0000","last_unstale":"2024-04-23T01:45:29.787991+0000","last_undegraded":"2024-04-23T01:45:29.787991+0000","last_fullsized":"2024-04-23T01:45:29.787991+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"17'183","reported_seq":196,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:31.495354+0000","last_change":"2024-04-23T01:45:16.859980+0000","last_active":"2024-04-23T01:45:31.495354+0000","last_peered":"2024-04-23T01:45:31.495354+0000","last_clean":"2024-04-23T01:45:31.495354+0000","last_became_active":"2024-04-23T01:45:16.855622+0000","last_became_peered":"2024-04-23T01:45:16.855622+0000","last_unstale":"2024-04-23T01:45:31.495354+0000","last_undegraded":"2024-04-23T01:45:31.495354+0000","last_fullsized":"2024-04-23T01:45:31.495354+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:18.596502+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:45:18.596502+0000","last_peered":"2024-04-23T01:45:18.596502+0000","last_clean":"2024-04-23T01:45:18.596502+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:45:18.596502+0000","last_undegraded":"2024-04-23T01:45:18.596502+0000","last_fullsized":"2024-04-23T01:45:18.596502+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"17'119","reported_seq":132,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:30.675784+0000","last_change":"2024-04-23T01:45:16.854641+0000","last_active":"2024-04-23T01:45:30.675784+0000","last_peered":"2024-04-23T01:45:30.675784+0000","last_clean":"2024-04-23T01:45:30.675784+0000","last_became_active":"2024-04-23T01:45:16.845783+0000","last_became_peered":"2024-04-23T01:45:16.845783+0000","last_unstale":"2024-04-23T01:45:30.675784+0000","last_undegraded":"2024-04-23T01:45:30.675784+0000","last_fullsized":"2024-04-23T01:45:30.675784+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:18.018544+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:45:18.018544+0000","last_peered":"2024-04-23T01:45:18.018544+0000","last_clean":"2024-04-23T01:45:18.018544+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:45:18.018544+0000","last_undegraded":"2024-04-23T01:45:18.018544+0000","last_fullsized":"2024-04-23T01:45:18.018544+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"17'227","reported_seq":240,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:31.554414+0000","last_change":"2024-04-23T01:45:16.845162+0000","last_active":"2024-04-23T01:45:31.554414+0000","last_peered":"2024-04-23T01:45:31.554414+0000","last_clean":"2024-04-23T01:45:31.554414+0000","last_became_active":"2024-04-23T01:45:16.829836+0000","last_became_peered":"2024-04-23T01:45:16.829836+0000","last_unstale":"2024-04-23T01:45:31.554414+0000","last_undegraded":"2024-04-23T01:45:31.554414+0000","last_fullsized":"2024-04-23T01:45:31.554414+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1867776,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":228,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:18.596986+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:45:18.596986+0000","last_peered":"2024-04-23T01:45:18.596986+0000","last_clean":"2024-04-23T01:45:18.596986+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:45:18.596986+0000","last_undegraded":"2024-04-23T01:45:18.596986+0000","last_fullsized":"2024-04-23T01:45:18.596986+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"17'119","reported_seq":132,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:31.513159+0000","last_change":"2024-04-23T01:45:16.885391+0000","last_active":"2024-04-23T01:45:31.513159+0000","last_peered":"2024-04-23T01:45:31.513159+0000","last_clean":"2024-04-23T01:45:31.513159+0000","last_became_active":"2024-04-23T01:45:16.880317+0000","last_became_peered":"2024-04-23T01:45:16.880317+0000","last_unstale":"2024-04-23T01:45:31.513159+0000","last_undegraded":"2024-04-23T01:45:31.513159+0000","last_fullsized":"2024-04-23T01:45:31.513159+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:17.724096+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:45:17.724096+0000","last_peered":"2024-04-23T01:45:17.724096+0000","last_clean":"2024-04-23T01:45:17.724096+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:45:17.724096+0000","last_undegraded":"2024-04-23T01:45:17.724096+0000","last_fullsized":"2024-04-23T01:45:17.724096+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":18309120,"num_objects":281,"num_object_clones":0,"num_object_copies":843,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2235,"num_write_kb":17880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":27447296,"data_stored":54894592,"data_compressed":2831193,"data_compressed_allocated":27447296,"data_compressed_original":54894592,"omap_allocated":0,"internal_metadata":0},"log_size":2219,"ondisk_log_size":2219,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705688,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":42444,"kb_used_data":7300,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94329396,"statfs":{"total":96636764160,"available":96593301504,"internally_reserved":0,"allocated":7475200,"data_stored":15172389,"data_compressed":779860,"data_compressed_allocated":7311360,"data_compressed_original":15114240,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705688,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36772,"kb_used_data":5788,"kb_used_omap":0,"kb_used_meta":30976,"kb_avail":94335068,"statfs":{"total":96636764160,"available":96599109632,"internally_reserved":0,"allocated":5926912,"data_stored":12075813,"data_compressed":620154,"data_compressed_allocated":5763072,"data_compressed_original":12017664,"omap_allocated":0,"internal_metadata":31719424},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705688,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37732,"kb_used_data":6748,"kb_used_omap":0,"kb_used_meta":30976,"kb_avail":94334108,"statfs":{"total":96636764160,"available":96598126592,"internally_reserved":0,"allocated":6909952,"data_stored":13574405,"data_compressed":697878,"data_compressed_allocated":6766592,"data_compressed_original":13533184,"omap_allocated":0,"internal_metadata":31719424},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705688,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36840,"kb_used_data":5856,"kb_used_omap":0,"kb_used_meta":30976,"kb_avail":94335000,"statfs":{"total":96636764160,"available":96599040000,"internally_reserved":0,"allocated":5996544,"data_stored":12215077,"data_compressed":627311,"data_compressed_allocated":5832704,"data_compressed_original":12156928,"omap_allocated":0,"internal_metadata":31719424},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6172672,"data_stored":12345344,"data_compressed":636697,"data_compressed_allocated":6172672,"data_compressed_original":12345344,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7376896,"data_stored":14753792,"data_compressed":760905,"data_compressed_allocated":7376896,"data_compressed_original":14753792,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6021120,"data_stored":12042240,"data_compressed":621080,"data_compressed_allocated":6021120,"data_compressed_original":12042240,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7876608,"data_stored":15753216,"data_compressed":812511,"data_compressed_allocated":7876608,"data_compressed_original":15753216,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:45:36.219 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:45:36.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:36.347+0000 7f7e26eed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:36.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:36.359+0000 7f7e26eed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:36.598 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:45:36.599 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:45:36.616 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-04-23T01:45:35.751867+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":18899488,"num_objects":283,"num_object_clones":0,"num_object_copies":849,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":2372,"num_write_kb":19774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2316,"ondisk_log_size":2316,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":153788,"kb_used_data":25692,"kb_used_omap":0,"kb_used_meta":128064,"kb_avail":377333572,"statfs":{"total":386547056640,"available":386389577728,"internally_reserved":0,"allocated":26308608,"data_stored":53037684,"data_compressed":2725203,"data_compressed_allocated":25673728,"data_compressed_original":52822016,"omap_allocated":0,"internal_metadata":131137536},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6619136,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":808,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.199254"},"pg_stats":[{"pgid":"3.f","version":"17'119","reported_seq":132,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:31.134257+0000","last_change":"2024-04-23T01:45:16.856566+0000","last_active":"2024-04-23T01:45:31.134257+0000","last_peered":"2024-04-23T01:45:31.134257+0000","last_clean":"2024-04-23T01:45:31.134257+0000","last_became_active":"2024-04-23T01:45:16.848194+0000","last_became_peered":"2024-04-23T01:45:16.848194+0000","last_unstale":"2024-04-23T01:45:31.134257+0000","last_undegraded":"2024-04-23T01:45:31.134257+0000","last_fullsized":"2024-04-23T01:45:31.134257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"17'189","reported_seq":202,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:31.571329+0000","last_change":"2024-04-23T01:45:16.906007+0000","last_active":"2024-04-23T01:45:31.571329+0000","last_peered":"2024-04-23T01:45:31.571329+0000","last_clean":"2024-04-23T01:45:31.571329+0000","last_became_active":"2024-04-23T01:45:16.901215+0000","last_became_peered":"2024-04-23T01:45:16.901215+0000","last_unstale":"2024-04-23T01:45:31.571329+0000","last_undegraded":"2024-04-23T01:45:31.571329+0000","last_fullsized":"2024-04-23T01:45:31.571329+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1556480,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":190,"num_write_kb":1520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"17'151","reported_seq":164,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:31.185910+0000","last_change":"2024-04-23T01:45:16.886628+0000","last_active":"2024-04-23T01:45:31.185910+0000","last_peered":"2024-04-23T01:45:31.185910+0000","last_clean":"2024-04-23T01:45:31.185910+0000","last_became_active":"2024-04-23T01:45:16.880681+0000","last_became_peered":"2024-04-23T01:45:16.880681+0000","last_unstale":"2024-04-23T01:45:31.185910+0000","last_undegraded":"2024-04-23T01:45:31.185910+0000","last_fullsized":"2024-04-23T01:45:31.185910+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"17'103","reported_seq":116,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:30.446620+0000","last_change":"2024-04-23T01:45:16.836330+0000","last_active":"2024-04-23T01:45:30.446620+0000","last_peered":"2024-04-23T01:45:30.446620+0000","last_clean":"2024-04-23T01:45:30.446620+0000","last_became_active":"2024-04-23T01:45:16.821451+0000","last_became_peered":"2024-04-23T01:45:16.821451+0000","last_unstale":"2024-04-23T01:45:30.446620+0000","last_undegraded":"2024-04-23T01:45:30.446620+0000","last_fullsized":"2024-04-23T01:45:30.446620+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"17'103","reported_seq":116,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:28.512016+0000","last_change":"2024-04-23T01:45:16.860867+0000","last_active":"2024-04-23T01:45:28.512016+0000","last_peered":"2024-04-23T01:45:28.512016+0000","last_clean":"2024-04-23T01:45:28.512016+0000","last_became_active":"2024-04-23T01:45:16.852271+0000","last_became_peered":"2024-04-23T01:45:16.852271+0000","last_unstale":"2024-04-23T01:45:28.512016+0000","last_undegraded":"2024-04-23T01:45:28.512016+0000","last_fullsized":"2024-04-23T01:45:28.512016+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"17'167","reported_seq":180,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:29.681197+0000","last_change":"2024-04-23T01:45:16.805452+0000","last_active":"2024-04-23T01:45:29.681197+0000","last_peered":"2024-04-23T01:45:29.681197+0000","last_clean":"2024-04-23T01:45:29.681197+0000","last_became_active":"2024-04-23T01:45:16.797805+0000","last_became_peered":"2024-04-23T01:45:16.797805+0000","last_unstale":"2024-04-23T01:45:29.681197+0000","last_undegraded":"2024-04-23T01:45:29.681197+0000","last_fullsized":"2024-04-23T01:45:29.681197+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":1344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"17'143","reported_seq":156,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:30.853936+0000","last_change":"2024-04-23T01:45:16.893894+0000","last_active":"2024-04-23T01:45:30.853936+0000","last_peered":"2024-04-23T01:45:30.853936+0000","last_clean":"2024-04-23T01:45:30.853936+0000","last_became_active":"2024-04-23T01:45:16.888673+0000","last_became_peered":"2024-04-23T01:45:16.888673+0000","last_unstale":"2024-04-23T01:45:30.853936+0000","last_undegraded":"2024-04-23T01:45:30.853936+0000","last_fullsized":"2024-04-23T01:45:30.853936+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"17'127","reported_seq":140,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:31.372315+0000","last_change":"2024-04-23T01:45:16.889162+0000","last_active":"2024-04-23T01:45:31.372315+0000","last_peered":"2024-04-23T01:45:31.372315+0000","last_clean":"2024-04-23T01:45:31.372315+0000","last_became_active":"2024-04-23T01:45:16.880761+0000","last_became_peered":"2024-04-23T01:45:16.880761+0000","last_unstale":"2024-04-23T01:45:31.372315+0000","last_undegraded":"2024-04-23T01:45:31.372315+0000","last_fullsized":"2024-04-23T01:45:31.372315+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":154,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:18.010968+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:45:18.010968+0000","last_peered":"2024-04-23T01:45:18.010968+0000","last_clean":"2024-04-23T01:45:18.010968+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:45:18.010968+0000","last_undegraded":"2024-04-23T01:45:18.010968+0000","last_fullsized":"2024-04-23T01:45:18.010968+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:17.738806+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:45:17.738806+0000","last_peered":"2024-04-23T01:45:17.738806+0000","last_clean":"2024-04-23T01:45:17.738806+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:45:17.738806+0000","last_undegraded":"2024-04-23T01:45:17.738806+0000","last_fullsized":"2024-04-23T01:45:17.738806+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"17'127","reported_seq":140,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:31.270601+0000","last_change":"2024-04-23T01:45:16.891787+0000","last_active":"2024-04-23T01:45:31.270601+0000","last_peered":"2024-04-23T01:45:31.270601+0000","last_clean":"2024-04-23T01:45:31.270601+0000","last_became_active":"2024-04-23T01:45:16.885404+0000","last_became_peered":"2024-04-23T01:45:16.885404+0000","last_unstale":"2024-04-23T01:45:31.270601+0000","last_undegraded":"2024-04-23T01:45:31.270601+0000","last_fullsized":"2024-04-23T01:45:31.270601+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:17.848366+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:45:17.848366+0000","last_peered":"2024-04-23T01:45:17.848366+0000","last_clean":"2024-04-23T01:45:17.848366+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:45:17.848366+0000","last_undegraded":"2024-04-23T01:45:17.848366+0000","last_fullsized":"2024-04-23T01:45:17.848366+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"17'135","reported_seq":148,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:30.637767+0000","last_change":"2024-04-23T01:45:16.919259+0000","last_active":"2024-04-23T01:45:30.637767+0000","last_peered":"2024-04-23T01:45:30.637767+0000","last_clean":"2024-04-23T01:45:30.637767+0000","last_became_active":"2024-04-23T01:45:16.912983+0000","last_became_peered":"2024-04-23T01:45:16.912983+0000","last_unstale":"2024-04-23T01:45:30.637767+0000","last_undegraded":"2024-04-23T01:45:30.637767+0000","last_fullsized":"2024-04-23T01:45:30.637767+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":136,"num_write_kb":1088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:17.722899+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:45:17.722899+0000","last_peered":"2024-04-23T01:45:17.722899+0000","last_clean":"2024-04-23T01:45:17.722899+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:45:17.722899+0000","last_undegraded":"2024-04-23T01:45:17.722899+0000","last_fullsized":"2024-04-23T01:45:17.722899+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"17'104","reported_seq":117,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:31.275664+0000","last_change":"2024-04-23T01:45:16.897089+0000","last_active":"2024-04-23T01:45:31.275664+0000","last_peered":"2024-04-23T01:45:31.275664+0000","last_clean":"2024-04-23T01:45:31.275664+0000","last_became_active":"2024-04-23T01:45:16.891776+0000","last_became_peered":"2024-04-23T01:45:16.891776+0000","last_unstale":"2024-04-23T01:45:31.275664+0000","last_undegraded":"2024-04-23T01:45:31.275664+0000","last_fullsized":"2024-04-23T01:45:31.275664+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":860160,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105,"num_write_kb":840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:17.846928+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:45:17.846928+0000","last_peered":"2024-04-23T01:45:17.846928+0000","last_clean":"2024-04-23T01:45:17.846928+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:45:17.846928+0000","last_undegraded":"2024-04-23T01:45:17.846928+0000","last_fullsized":"2024-04-23T01:45:17.846928+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"17'103","reported_seq":116,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:29.787991+0000","last_change":"2024-04-23T01:45:16.823909+0000","last_active":"2024-04-23T01:45:29.787991+0000","last_peered":"2024-04-23T01:45:29.787991+0000","last_clean":"2024-04-23T01:45:29.787991+0000","last_became_active":"2024-04-23T01:45:16.815467+0000","last_became_peered":"2024-04-23T01:45:16.815467+0000","last_unstale":"2024-04-23T01:45:29.787991+0000","last_undegraded":"2024-04-23T01:45:29.787991+0000","last_fullsized":"2024-04-23T01:45:29.787991+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"17'183","reported_seq":196,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:31.495354+0000","last_change":"2024-04-23T01:45:16.859980+0000","last_active":"2024-04-23T01:45:31.495354+0000","last_peered":"2024-04-23T01:45:31.495354+0000","last_clean":"2024-04-23T01:45:31.495354+0000","last_became_active":"2024-04-23T01:45:16.855622+0000","last_became_peered":"2024-04-23T01:45:16.855622+0000","last_unstale":"2024-04-23T01:45:31.495354+0000","last_undegraded":"2024-04-23T01:45:31.495354+0000","last_fullsized":"2024-04-23T01:45:31.495354+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:18.596502+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:45:18.596502+0000","last_peered":"2024-04-23T01:45:18.596502+0000","last_clean":"2024-04-23T01:45:18.596502+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:45:18.596502+0000","last_undegraded":"2024-04-23T01:45:18.596502+0000","last_fullsized":"2024-04-23T01:45:18.596502+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"17'119","reported_seq":132,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:30.675784+0000","last_change":"2024-04-23T01:45:16.854641+0000","last_active":"2024-04-23T01:45:30.675784+0000","last_peered":"2024-04-23T01:45:30.675784+0000","last_clean":"2024-04-23T01:45:30.675784+0000","last_became_active":"2024-04-23T01:45:16.845783+0000","last_became_peered":"2024-04-23T01:45:16.845783+0000","last_unstale":"2024-04-23T01:45:30.675784+0000","last_undegraded":"2024-04-23T01:45:30.675784+0000","last_fullsized":"2024-04-23T01:45:30.675784+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:18.018544+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:45:18.018544+0000","last_peered":"2024-04-23T01:45:18.018544+0000","last_clean":"2024-04-23T01:45:18.018544+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:45:18.018544+0000","last_undegraded":"2024-04-23T01:45:18.018544+0000","last_fullsized":"2024-04-23T01:45:18.018544+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"17'227","reported_seq":240,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:31.554414+0000","last_change":"2024-04-23T01:45:16.845162+0000","last_active":"2024-04-23T01:45:31.554414+0000","last_peered":"2024-04-23T01:45:31.554414+0000","last_clean":"2024-04-23T01:45:31.554414+0000","last_became_active":"2024-04-23T01:45:16.829836+0000","last_became_peered":"2024-04-23T01:45:16.829836+0000","last_unstale":"2024-04-23T01:45:31.554414+0000","last_undegraded":"2024-04-23T01:45:31.554414+0000","last_fullsized":"2024-04-23T01:45:31.554414+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1867776,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":228,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:18.596986+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:45:18.596986+0000","last_peered":"2024-04-23T01:45:18.596986+0000","last_clean":"2024-04-23T01:45:18.596986+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:45:18.596986+0000","last_undegraded":"2024-04-23T01:45:18.596986+0000","last_fullsized":"2024-04-23T01:45:18.596986+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"17'119","reported_seq":132,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:31.513159+0000","last_change":"2024-04-23T01:45:16.885391+0000","last_active":"2024-04-23T01:45:31.513159+0000","last_peered":"2024-04-23T01:45:31.513159+0000","last_clean":"2024-04-23T01:45:31.513159+0000","last_became_active":"2024-04-23T01:45:16.880317+0000","last_became_peered":"2024-04-23T01:45:16.880317+0000","last_unstale":"2024-04-23T01:45:31.513159+0000","last_undegraded":"2024-04-23T01:45:31.513159+0000","last_fullsized":"2024-04-23T01:45:31.513159+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-23T01:45:17.724096+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:45:17.724096+0000","last_peered":"2024-04-23T01:45:17.724096+0000","last_clean":"2024-04-23T01:45:17.724096+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:45:17.724096+0000","last_undegraded":"2024-04-23T01:45:17.724096+0000","last_fullsized":"2024-04-23T01:45:17.724096+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":18309120,"num_objects":281,"num_object_clones":0,"num_object_copies":843,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2235,"num_write_kb":17880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":27447296,"data_stored":54894592,"data_compressed":2831193,"data_compressed_allocated":27447296,"data_compressed_original":54894592,"omap_allocated":0,"internal_metadata":0},"log_size":2219,"ondisk_log_size":2219,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705688,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":42444,"kb_used_data":7300,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94329396,"statfs":{"total":96636764160,"available":96593301504,"internally_reserved":0,"allocated":7475200,"data_stored":15172389,"data_compressed":779860,"data_compressed_allocated":7311360,"data_compressed_original":15114240,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705688,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36772,"kb_used_data":5788,"kb_used_omap":0,"kb_used_meta":30976,"kb_avail":94335068,"statfs":{"total":96636764160,"available":96599109632,"internally_reserved":0,"allocated":5926912,"data_stored":12075813,"data_compressed":620154,"data_compressed_allocated":5763072,"data_compressed_original":12017664,"omap_allocated":0,"internal_metadata":31719424},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705688,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37732,"kb_used_data":6748,"kb_used_omap":0,"kb_used_meta":30976,"kb_avail":94334108,"statfs":{"total":96636764160,"available":96598126592,"internally_reserved":0,"allocated":6909952,"data_stored":13574405,"data_compressed":697878,"data_compressed_allocated":6766592,"data_compressed_original":13533184,"omap_allocated":0,"internal_metadata":31719424},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705688,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36840,"kb_used_data":5856,"kb_used_omap":0,"kb_used_meta":30976,"kb_avail":94335000,"statfs":{"total":96636764160,"available":96599040000,"internally_reserved":0,"allocated":5996544,"data_stored":12215077,"data_compressed":627311,"data_compressed_allocated":5832704,"data_compressed_original":12156928,"omap_allocated":0,"internal_metadata":31719424},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6172672,"data_stored":12345344,"data_compressed":636697,"data_compressed_allocated":6172672,"data_compressed_original":12345344,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7376896,"data_stored":14753792,"data_compressed":760905,"data_compressed_allocated":7376896,"data_compressed_original":14753792,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6021120,"data_stored":12042240,"data_compressed":621080,"data_compressed_allocated":6021120,"data_compressed_original":12042240,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7876608,"data_stored":15753216,"data_compressed":812511,"data_compressed_allocated":7876608,"data_compressed_original":15753216,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:45:36.617 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-23T01:45:36.617 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:45:36.618 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:45:36.618 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:45:36.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:36.766+0000 7fda8c154640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:36.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:36.776+0000 7fda8c154640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:36.790 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 19 16 3036 3020 1.24167 1.23438 0.199941 0.100332 2024-04-23T01:45:36.795 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:45:36.795 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:45:36.795 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:45:36.795 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:45:36.796 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:45:36.796 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:45:36.796 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:45:36.796 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:45:36.796 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:45:36.796 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:45:36.796 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:45:36.796 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:45:36.799 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:45:36.800 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:45:36.800 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:45:36.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:36.928+0000 7f4e2646d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:36.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:36.940+0000 7f4e2646d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:36.955 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:45:36.956 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:45:36.956 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:45:36.956 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:45:36.956 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:45:36.956 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:45:36.956 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:45:36.956 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:45:36.956 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:45:36.956 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:45:36.956 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:45:36.956 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:45:36.959 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:45:36.959 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:45:36.960 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:45:37.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:37.071+0000 7f0577e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:37.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:37.085+0000 7f0577e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:37.106 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:45:37.106 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:45:37.106 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:45:37.106 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:45:37.106 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:45:37.106 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:45:37.106 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:45:37.106 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:45:37.106 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:45:37.107 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:45:37.107 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:45:37.107 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:45:37.109 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:45:37.109 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:45:37.110 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:45:37.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:37.254+0000 7f6cb4024640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:37.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:37.266+0000 7f6cb4024640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:37.291 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:45:37.291 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:45:37.291 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:45:37.291 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:45:37.291 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:45:37.291 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:45:37.291 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:45:37.291 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:45:37.291 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:45:37.291 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:45:37.292 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:45:37.292 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:45:37.294 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:45:37.294 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:45:37.790 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:45:37.793098+0000 min lat: 0.0153353 max lat: 0.332121 avg lat: 0.100621 2024-04-23T01:45:37.791 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:45:37.791 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 20 16 3186 3170 1.23818 1.17188 0.123806 0.100621 2024-04-23T01:45:38.041 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:45:38.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:38.174+0000 7f29ef954640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:38.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:38.181+0000 7f29ef954640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:38.791 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 21 16 3355 3339 1.24208 1.32031 0.109058 0.100395 2024-04-23T01:45:39.791 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 22 16 3523 3507 1.24528 1.3125 0.092052 0.100109 2024-04-23T01:45:40.409 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:45:40.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:40.547+0000 7f37f0ead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:40.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:40.553+0000 7f37f0ead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:40.791 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 23 16 3676 3660 1.2431 1.19531 0.0781164 0.100273 2024-04-23T01:45:41.791 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 24 16 3840 3824 1.24469 1.28125 0.0757481 0.100228 2024-04-23T01:45:42.294 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:45:42.295 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:45:42.295 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:45:42.295 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:45:42.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:42.452+0000 7fe66ea6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:42.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:42.464+0000 7fe66ea6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:42.521 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:45:42.521 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:45:42.521 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:45:42.521 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:45:42.522 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:45:42.522 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:45:42.522 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:45:42.522 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:45:42.522 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:45:42.522 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:45:42.522 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:45:42.522 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:45:42.525 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:45:42.526 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:45:42.526 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:45:42.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:42.647+0000 7fb28506d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:42.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:42.659+0000 7fb28506d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:42.681 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:45:42.681 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:45:42.681 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:45:42.681 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:45:42.681 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:45:42.681 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:45:42.681 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:45:42.681 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:45:42.681 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:45:42.681 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:45:42.681 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:45:42.681 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:45:42.684 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:45:42.684 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:45:42.684 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:45:42.774 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:45:42.791 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 25 16 4001 3985 1.24521 1.25781 0.148063 0.100052 2024-04-23T01:45:42.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:42.823+0000 7fe3aae2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:42.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:42.834+0000 7fe3aae2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:42.855 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:45:42.855 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:45:42.855 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:45:42.855 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:45:42.856 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:45:42.856 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:45:42.856 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:45:42.856 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:45:42.856 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:45:42.856 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:45:42.856 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:45:42.856 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:45:42.860 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:45:42.860 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:45:42.860 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:45:42.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:42.904+0000 7f867e0ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:42.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:42.910+0000 7f867e0ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:43.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:43.040+0000 7f2d5479d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:43.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:43.054+0000 7f2d5479d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:43.092 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:45:43.093 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:45:43.093 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:45:43.093 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:45:43.093 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:45:43.093 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:45:43.093 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:45:43.093 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:45:43.093 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:45:43.093 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:45:43.093 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:45:43.093 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:45:43.096 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:45:43.096 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:45:43.791 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 26 16 4159 4143 1.24478 1.23438 0.145403 0.100203 2024-04-23T01:45:44.791 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 27 16 4317 4301 1.2444 1.23438 0.0301165 0.100246 2024-04-23T01:45:45.083 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:45:45.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:45.222+0000 7fafeb79d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:45.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:45.228+0000 7fafeb79d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:45.791 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 28 16 4469 4453 1.24236 1.1875 0.201929 0.100381 2024-04-23T01:45:46.791 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 29 16 4635 4619 1.24423 1.29688 0.135099 0.100215 2024-04-23T01:45:47.481 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:45:47.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:47.607+0000 7fd5e8cad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:47.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:47.613+0000 7fd5e8cad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:47.791 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 30 16 4786 4770 1.24208 1.17969 0.219221 0.100297 2024-04-23T01:45:48.096 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:45:48.097 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:45:48.097 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:45:48.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:48.220+0000 7f317902d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:48.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:48.229+0000 7f317902d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:48.447 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:45:48.448 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:45:48.464 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-04-23T01:45:47.555681+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":38183456,"num_objects":578,"num_object_clones":0,"num_object_copies":1734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":4726,"num_write_kb":38606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4670,"ondisk_log_size":4670,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":215352,"kb_used_data":53972,"kb_used_omap":0,"kb_used_meta":161344,"kb_avail":377272008,"statfs":{"total":386547056640,"available":386326536192,"internally_reserved":0,"allocated":55267328,"data_stored":110744900,"data_compressed":5702277,"data_compressed_allocated":54501376,"data_compressed_original":110477312,"omap_allocated":0,"internal_metadata":165216256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19283968,"num_objects":295,"num_object_clones":0,"num_object_copies":885,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2354,"num_write_kb":18832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.803813"},"pg_stats":[{"pgid":"3.f","version":"21'247","reported_seq":270,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:46.202739+0000","last_change":"2024-04-23T01:45:16.856566+0000","last_active":"2024-04-23T01:45:46.202739+0000","last_peered":"2024-04-23T01:45:46.202739+0000","last_clean":"2024-04-23T01:45:46.202739+0000","last_became_active":"2024-04-23T01:45:16.848194+0000","last_became_peered":"2024-04-23T01:45:16.848194+0000","last_unstale":"2024-04-23T01:45:46.202739+0000","last_undegraded":"2024-04-23T01:45:46.202739+0000","last_fullsized":"2024-04-23T01:45:46.202739+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"21'317","reported_seq":342,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:46.572808+0000","last_change":"2024-04-23T01:45:16.906007+0000","last_active":"2024-04-23T01:45:46.572808+0000","last_peered":"2024-04-23T01:45:46.572808+0000","last_clean":"2024-04-23T01:45:46.572808+0000","last_became_active":"2024-04-23T01:45:16.901215+0000","last_became_peered":"2024-04-23T01:45:16.901215+0000","last_unstale":"2024-04-23T01:45:46.572808+0000","last_undegraded":"2024-04-23T01:45:46.572808+0000","last_fullsized":"2024-04-23T01:45:46.572808+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2605056,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":318,"num_write_kb":2544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"21'327","reported_seq":350,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:46.496336+0000","last_change":"2024-04-23T01:45:16.886628+0000","last_active":"2024-04-23T01:45:46.496336+0000","last_peered":"2024-04-23T01:45:46.496336+0000","last_clean":"2024-04-23T01:45:46.496336+0000","last_became_active":"2024-04-23T01:45:16.880681+0000","last_became_peered":"2024-04-23T01:45:16.880681+0000","last_unstale":"2024-04-23T01:45:46.496336+0000","last_undegraded":"2024-04-23T01:45:46.496336+0000","last_fullsized":"2024-04-23T01:45:46.496336+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2686976,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":328,"num_write_kb":2624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"21'247","reported_seq":270,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:45.227204+0000","last_change":"2024-04-23T01:45:16.836330+0000","last_active":"2024-04-23T01:45:45.227204+0000","last_peered":"2024-04-23T01:45:45.227204+0000","last_clean":"2024-04-23T01:45:45.227204+0000","last_became_active":"2024-04-23T01:45:16.821451+0000","last_became_peered":"2024-04-23T01:45:16.821451+0000","last_unstale":"2024-04-23T01:45:45.227204+0000","last_undegraded":"2024-04-23T01:45:45.227204+0000","last_fullsized":"2024-04-23T01:45:45.227204+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"21'231","reported_seq":254,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:46.493627+0000","last_change":"2024-04-23T01:45:16.860867+0000","last_active":"2024-04-23T01:45:46.493627+0000","last_peered":"2024-04-23T01:45:46.493627+0000","last_clean":"2024-04-23T01:45:46.493627+0000","last_became_active":"2024-04-23T01:45:16.852271+0000","last_became_peered":"2024-04-23T01:45:16.852271+0000","last_unstale":"2024-04-23T01:45:46.493627+0000","last_undegraded":"2024-04-23T01:45:46.493627+0000","last_fullsized":"2024-04-23T01:45:46.493627+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1900544,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":232,"num_write_kb":1856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"21'287","reported_seq":310,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:45.938211+0000","last_change":"2024-04-23T01:45:16.805452+0000","last_active":"2024-04-23T01:45:45.938211+0000","last_peered":"2024-04-23T01:45:45.938211+0000","last_clean":"2024-04-23T01:45:45.938211+0000","last_became_active":"2024-04-23T01:45:16.797805+0000","last_became_peered":"2024-04-23T01:45:16.797805+0000","last_unstale":"2024-04-23T01:45:45.938211+0000","last_undegraded":"2024-04-23T01:45:45.938211+0000","last_fullsized":"2024-04-23T01:45:45.938211+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"21'314","reported_seq":337,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:46.563778+0000","last_change":"2024-04-23T01:45:16.893894+0000","last_active":"2024-04-23T01:45:46.563778+0000","last_peered":"2024-04-23T01:45:46.563778+0000","last_clean":"2024-04-23T01:45:46.563778+0000","last_became_active":"2024-04-23T01:45:16.888673+0000","last_became_peered":"2024-04-23T01:45:16.888673+0000","last_unstale":"2024-04-23T01:45:46.563778+0000","last_undegraded":"2024-04-23T01:45:46.563778+0000","last_fullsized":"2024-04-23T01:45:46.563778+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2580480,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":315,"num_write_kb":2520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"21'272","reported_seq":295,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:45.667131+0000","last_change":"2024-04-23T01:45:16.889162+0000","last_active":"2024-04-23T01:45:45.667131+0000","last_peered":"2024-04-23T01:45:45.667131+0000","last_clean":"2024-04-23T01:45:45.667131+0000","last_became_active":"2024-04-23T01:45:16.880761+0000","last_became_peered":"2024-04-23T01:45:16.880761+0000","last_unstale":"2024-04-23T01:45:45.667131+0000","last_undegraded":"2024-04-23T01:45:45.667131+0000","last_fullsized":"2024-04-23T01:45:45.667131+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2236416,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":2184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":164,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:40.713962+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:45:40.713962+0000","last_peered":"2024-04-23T01:45:40.713962+0000","last_clean":"2024-04-23T01:45:40.713962+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:45:40.713962+0000","last_undegraded":"2024-04-23T01:45:40.713962+0000","last_fullsized":"2024-04-23T01:45:40.713962+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:40.867514+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:45:40.867514+0000","last_peered":"2024-04-23T01:45:40.867514+0000","last_clean":"2024-04-23T01:45:40.867514+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:45:40.867514+0000","last_undegraded":"2024-04-23T01:45:40.867514+0000","last_fullsized":"2024-04-23T01:45:40.867514+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"21'255","reported_seq":278,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:46.201915+0000","last_change":"2024-04-23T01:45:16.891787+0000","last_active":"2024-04-23T01:45:46.201915+0000","last_peered":"2024-04-23T01:45:46.201915+0000","last_clean":"2024-04-23T01:45:46.201915+0000","last_became_active":"2024-04-23T01:45:16.885404+0000","last_became_peered":"2024-04-23T01:45:16.885404+0000","last_unstale":"2024-04-23T01:45:46.201915+0000","last_undegraded":"2024-04-23T01:45:46.201915+0000","last_fullsized":"2024-04-23T01:45:46.201915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":32,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":256,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:40.842025+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:45:40.842025+0000","last_peered":"2024-04-23T01:45:40.842025+0000","last_clean":"2024-04-23T01:45:40.842025+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:45:40.842025+0000","last_undegraded":"2024-04-23T01:45:40.842025+0000","last_fullsized":"2024-04-23T01:45:40.842025+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"21'239","reported_seq":262,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:45.716467+0000","last_change":"2024-04-23T01:45:16.919259+0000","last_active":"2024-04-23T01:45:45.716467+0000","last_peered":"2024-04-23T01:45:45.716467+0000","last_clean":"2024-04-23T01:45:45.716467+0000","last_became_active":"2024-04-23T01:45:16.912983+0000","last_became_peered":"2024-04-23T01:45:16.912983+0000","last_unstale":"2024-04-23T01:45:45.716467+0000","last_undegraded":"2024-04-23T01:45:45.716467+0000","last_fullsized":"2024-04-23T01:45:45.716467+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":240,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:40.855832+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:45:40.855832+0000","last_peered":"2024-04-23T01:45:40.855832+0000","last_clean":"2024-04-23T01:45:40.855832+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:45:40.855832+0000","last_undegraded":"2024-04-23T01:45:40.855832+0000","last_fullsized":"2024-04-23T01:45:40.855832+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"21'303","reported_seq":326,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:46.289632+0000","last_change":"2024-04-23T01:45:16.897089+0000","last_active":"2024-04-23T01:45:46.289632+0000","last_peered":"2024-04-23T01:45:46.289632+0000","last_clean":"2024-04-23T01:45:46.289632+0000","last_became_active":"2024-04-23T01:45:16.891776+0000","last_became_peered":"2024-04-23T01:45:16.891776+0000","last_unstale":"2024-04-23T01:45:46.289632+0000","last_undegraded":"2024-04-23T01:45:46.289632+0000","last_fullsized":"2024-04-23T01:45:46.289632+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490368,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:40.840035+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:45:40.840035+0000","last_peered":"2024-04-23T01:45:40.840035+0000","last_clean":"2024-04-23T01:45:40.840035+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:45:40.840035+0000","last_undegraded":"2024-04-23T01:45:40.840035+0000","last_fullsized":"2024-04-23T01:45:40.840035+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"21'271","reported_seq":296,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:45.527475+0000","last_change":"2024-04-23T01:45:16.823909+0000","last_active":"2024-04-23T01:45:45.527475+0000","last_peered":"2024-04-23T01:45:45.527475+0000","last_clean":"2024-04-23T01:45:45.527475+0000","last_became_active":"2024-04-23T01:45:16.815467+0000","last_became_peered":"2024-04-23T01:45:16.815467+0000","last_unstale":"2024-04-23T01:45:45.527475+0000","last_undegraded":"2024-04-23T01:45:45.527475+0000","last_fullsized":"2024-04-23T01:45:45.527475+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":272,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"21'311","reported_seq":334,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:44.735331+0000","last_change":"2024-04-23T01:45:16.859980+0000","last_active":"2024-04-23T01:45:44.735331+0000","last_peered":"2024-04-23T01:45:44.735331+0000","last_clean":"2024-04-23T01:45:44.735331+0000","last_became_active":"2024-04-23T01:45:16.855622+0000","last_became_peered":"2024-04-23T01:45:16.855622+0000","last_unstale":"2024-04-23T01:45:44.735331+0000","last_undegraded":"2024-04-23T01:45:44.735331+0000","last_fullsized":"2024-04-23T01:45:44.735331+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2555904,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":2496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:35.774643+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:45:35.774643+0000","last_peered":"2024-04-23T01:45:35.774643+0000","last_clean":"2024-04-23T01:45:35.774643+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:45:35.774643+0000","last_undegraded":"2024-04-23T01:45:35.774643+0000","last_fullsized":"2024-04-23T01:45:35.774643+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"21'247","reported_seq":272,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:46.386848+0000","last_change":"2024-04-23T01:45:16.854641+0000","last_active":"2024-04-23T01:45:46.386848+0000","last_peered":"2024-04-23T01:45:46.386848+0000","last_clean":"2024-04-23T01:45:46.386848+0000","last_became_active":"2024-04-23T01:45:16.845783+0000","last_became_peered":"2024-04-23T01:45:16.845783+0000","last_unstale":"2024-04-23T01:45:46.386848+0000","last_undegraded":"2024-04-23T01:45:46.386848+0000","last_fullsized":"2024-04-23T01:45:46.386848+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":31,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:40.725302+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:45:40.725302+0000","last_peered":"2024-04-23T01:45:40.725302+0000","last_clean":"2024-04-23T01:45:40.725302+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:45:40.725302+0000","last_undegraded":"2024-04-23T01:45:40.725302+0000","last_fullsized":"2024-04-23T01:45:40.725302+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"21'394","reported_seq":417,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:46.529721+0000","last_change":"2024-04-23T01:45:16.845162+0000","last_active":"2024-04-23T01:45:46.529721+0000","last_peered":"2024-04-23T01:45:46.529721+0000","last_clean":"2024-04-23T01:45:46.529721+0000","last_became_active":"2024-04-23T01:45:16.829836+0000","last_became_peered":"2024-04-23T01:45:16.829836+0000","last_unstale":"2024-04-23T01:45:46.529721+0000","last_undegraded":"2024-04-23T01:45:46.529721+0000","last_fullsized":"2024-04-23T01:45:46.529721+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3235840,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":395,"num_write_kb":3160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":31,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:35.776533+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:45:35.776533+0000","last_peered":"2024-04-23T01:45:35.776533+0000","last_clean":"2024-04-23T01:45:35.776533+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:45:35.776533+0000","last_undegraded":"2024-04-23T01:45:35.776533+0000","last_fullsized":"2024-04-23T01:45:35.776533+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"21'311","reported_seq":336,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:46.491007+0000","last_change":"2024-04-23T01:45:16.885391+0000","last_active":"2024-04-23T01:45:46.491007+0000","last_peered":"2024-04-23T01:45:46.491007+0000","last_clean":"2024-04-23T01:45:46.491007+0000","last_became_active":"2024-04-23T01:45:16.880317+0000","last_became_peered":"2024-04-23T01:45:16.880317+0000","last_unstale":"2024-04-23T01:45:46.491007+0000","last_undegraded":"2024-04-23T01:45:46.491007+0000","last_fullsized":"2024-04-23T01:45:46.491007+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2555904,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":2496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:40.858645+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:45:40.858645+0000","last_peered":"2024-04-23T01:45:40.858645+0000","last_clean":"2024-04-23T01:45:40.858645+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:45:40.858645+0000","last_undegraded":"2024-04-23T01:45:40.858645+0000","last_fullsized":"2024-04-23T01:45:40.858645+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":37593088,"num_objects":576,"num_object_clones":0,"num_object_copies":1728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4589,"num_write_kb":36712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":56377344,"data_stored":112754688,"data_compressed":5818842,"data_compressed_allocated":56377344,"data_compressed_original":112754688,"omap_allocated":0,"internal_metadata":0},"log_size":4573,"ondisk_log_size":4573,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705703,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59032,"kb_used_data":15568,"kb_used_omap":0,"kb_used_meta":43456,"kb_avail":94312808,"statfs":{"total":96636764160,"available":96576315392,"internally_reserved":0,"allocated":15941632,"data_stored":32052697,"data_compressed":1650817,"data_compressed_allocated":15745024,"data_compressed_original":31981568,"omap_allocated":0,"internal_metadata":44498944},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705703,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51640,"kb_used_data":12332,"kb_used_omap":0,"kb_used_meta":39296,"kb_avail":94320200,"statfs":{"total":96636764160,"available":96583884800,"internally_reserved":0,"allocated":12627968,"data_stored":25425369,"data_compressed":1308798,"data_compressed_allocated":12431360,"data_compressed_original":25354240,"omap_allocated":0,"internal_metadata":40239104},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705703,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":53316,"kb_used_data":14012,"kb_used_omap":0,"kb_used_meta":39296,"kb_avail":94318524,"statfs":{"total":96636764160,"available":96582168576,"internally_reserved":0,"allocated":14348288,"data_stored":28398521,"data_compressed":1462662,"data_compressed_allocated":14172160,"data_compressed_original":28344320,"omap_allocated":0,"internal_metadata":40239104},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51364,"kb_used_data":12060,"kb_used_omap":0,"kb_used_meta":39296,"kb_avail":94320476,"statfs":{"total":96636764160,"available":96584167424,"internally_reserved":0,"allocated":12349440,"data_stored":24868313,"data_compressed":1280000,"data_compressed_allocated":12152832,"data_compressed_original":24797184,"omap_allocated":0,"internal_metadata":40239104},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12574720,"data_stored":25149440,"data_compressed":1297846,"data_compressed_allocated":12574720,"data_compressed_original":25149440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14700544,"data_stored":29401088,"data_compressed":1517229,"data_compressed_allocated":14700544,"data_compressed_original":29401088,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12779520,"data_stored":25559040,"data_compressed":1319030,"data_compressed_allocated":12779520,"data_compressed_original":25559040,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16322560,"data_stored":32645120,"data_compressed":1684737,"data_compressed_allocated":16322560,"data_compressed_original":32645120,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:45:48.792 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 31 16 4945 4929 1.24208 1.24219 0.102867 0.100466 2024-04-23T01:45:49.781 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:45:49.791 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 32 16 5108 5092 1.24306 1.27344 0.116555 0.100383 2024-04-23T01:45:49.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:49.908+0000 7f6799f5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:49.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:49.915+0000 7f6799f5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:50.792 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 33 16 5271 5255 1.24397 1.27344 0.143225 0.100265 2024-04-23T01:45:51.792 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 34 16 5426 5410 1.243 1.21094 0.12708 0.100377 2024-04-23T01:45:52.181 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:45:52.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:52.311+0000 7fe087b54640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:52.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:52.317+0000 7fe087b54640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:52.792 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 35 16 5569 5553 1.2394 1.11719 0.207956 0.10065 2024-04-23T01:45:53.466 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:45:53.466 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:45:53.466 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:45:53.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:53.601+0000 7f4f2695d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:53.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:53.611+0000 7f4f2695d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:53.792 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 36 16 5717 5701 1.23709 1.15625 0.124327 0.100838 2024-04-23T01:45:53.841 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:45:53.841 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:45:53.855 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-04-23T01:45:53.556854+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":44663328,"num_objects":676,"num_object_clones":0,"num_object_copies":2028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":5517,"num_write_kb":44934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5461,"ondisk_log_size":5461,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":195904,"kb_used_data":63648,"kb_used_omap":0,"kb_used_meta":132224,"kb_avail":377291456,"statfs":{"total":386547056640,"available":386346450944,"internally_reserved":0,"allocated":65175552,"data_stored":130561348,"data_compressed":6725514,"data_compressed_allocated":64409600,"data_compressed_original":130293760,"omap_allocated":0,"internal_metadata":135397376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19554304,"num_objects":299,"num_object_clones":0,"num_object_copies":897,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2387,"num_write_kb":19096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002273"},"pg_stats":[{"pgid":"3.f","version":"21'271","reported_seq":294,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:50.925745+0000","last_change":"2024-04-23T01:45:16.856566+0000","last_active":"2024-04-23T01:45:50.925745+0000","last_peered":"2024-04-23T01:45:50.925745+0000","last_clean":"2024-04-23T01:45:50.925745+0000","last_became_active":"2024-04-23T01:45:16.848194+0000","last_became_peered":"2024-04-23T01:45:16.848194+0000","last_unstale":"2024-04-23T01:45:50.925745+0000","last_undegraded":"2024-04-23T01:45:50.925745+0000","last_fullsized":"2024-04-23T01:45:50.925745+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":272,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"21'385","reported_seq":410,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:51.548898+0000","last_change":"2024-04-23T01:45:16.906007+0000","last_active":"2024-04-23T01:45:51.548898+0000","last_peered":"2024-04-23T01:45:51.548898+0000","last_clean":"2024-04-23T01:45:51.548898+0000","last_became_active":"2024-04-23T01:45:16.901215+0000","last_became_peered":"2024-04-23T01:45:16.901215+0000","last_unstale":"2024-04-23T01:45:51.548898+0000","last_undegraded":"2024-04-23T01:45:51.548898+0000","last_fullsized":"2024-04-23T01:45:51.548898+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3162112,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":386,"num_write_kb":3088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"21'343","reported_seq":366,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:48.439581+0000","last_change":"2024-04-23T01:45:16.886628+0000","last_active":"2024-04-23T01:45:48.439581+0000","last_peered":"2024-04-23T01:45:48.439581+0000","last_clean":"2024-04-23T01:45:48.439581+0000","last_became_active":"2024-04-23T01:45:16.880681+0000","last_became_peered":"2024-04-23T01:45:16.880681+0000","last_unstale":"2024-04-23T01:45:48.439581+0000","last_undegraded":"2024-04-23T01:45:48.439581+0000","last_fullsized":"2024-04-23T01:45:48.439581+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":344,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"21'343","reported_seq":366,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:51.417903+0000","last_change":"2024-04-23T01:45:16.836330+0000","last_active":"2024-04-23T01:45:51.417903+0000","last_peered":"2024-04-23T01:45:51.417903+0000","last_clean":"2024-04-23T01:45:51.417903+0000","last_became_active":"2024-04-23T01:45:16.821451+0000","last_became_peered":"2024-04-23T01:45:16.821451+0000","last_unstale":"2024-04-23T01:45:51.417903+0000","last_undegraded":"2024-04-23T01:45:51.417903+0000","last_fullsized":"2024-04-23T01:45:51.417903+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":344,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"21'255","reported_seq":278,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:49.649618+0000","last_change":"2024-04-23T01:45:16.860867+0000","last_active":"2024-04-23T01:45:49.649618+0000","last_peered":"2024-04-23T01:45:49.649618+0000","last_clean":"2024-04-23T01:45:49.649618+0000","last_became_active":"2024-04-23T01:45:16.852271+0000","last_became_peered":"2024-04-23T01:45:16.852271+0000","last_unstale":"2024-04-23T01:45:49.649618+0000","last_undegraded":"2024-04-23T01:45:49.649618+0000","last_fullsized":"2024-04-23T01:45:49.649618+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":32,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":256,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"21'351","reported_seq":374,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:50.721678+0000","last_change":"2024-04-23T01:45:16.805452+0000","last_active":"2024-04-23T01:45:50.721678+0000","last_peered":"2024-04-23T01:45:50.721678+0000","last_clean":"2024-04-23T01:45:50.721678+0000","last_became_active":"2024-04-23T01:45:16.797805+0000","last_became_peered":"2024-04-23T01:45:16.797805+0000","last_unstale":"2024-04-23T01:45:50.721678+0000","last_undegraded":"2024-04-23T01:45:50.721678+0000","last_fullsized":"2024-04-23T01:45:50.721678+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883584,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":352,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"21'383","reported_seq":406,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:51.538206+0000","last_change":"2024-04-23T01:45:16.893894+0000","last_active":"2024-04-23T01:45:51.538206+0000","last_peered":"2024-04-23T01:45:51.538206+0000","last_clean":"2024-04-23T01:45:51.538206+0000","last_became_active":"2024-04-23T01:45:16.888673+0000","last_became_peered":"2024-04-23T01:45:16.888673+0000","last_unstale":"2024-04-23T01:45:51.538206+0000","last_undegraded":"2024-04-23T01:45:51.538206+0000","last_fullsized":"2024-04-23T01:45:51.538206+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":384,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"21'311","reported_seq":334,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:50.864569+0000","last_change":"2024-04-23T01:45:16.889162+0000","last_active":"2024-04-23T01:45:50.864569+0000","last_peered":"2024-04-23T01:45:50.864569+0000","last_clean":"2024-04-23T01:45:50.864569+0000","last_became_active":"2024-04-23T01:45:16.880761+0000","last_became_peered":"2024-04-23T01:45:16.880761+0000","last_unstale":"2024-04-23T01:45:50.864569+0000","last_undegraded":"2024-04-23T01:45:50.864569+0000","last_fullsized":"2024-04-23T01:45:50.864569+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2555904,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":2496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":164,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:40.713962+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:45:40.713962+0000","last_peered":"2024-04-23T01:45:40.713962+0000","last_clean":"2024-04-23T01:45:40.713962+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:45:40.713962+0000","last_undegraded":"2024-04-23T01:45:40.713962+0000","last_fullsized":"2024-04-23T01:45:40.713962+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:40.867514+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:45:40.867514+0000","last_peered":"2024-04-23T01:45:40.867514+0000","last_clean":"2024-04-23T01:45:40.867514+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:45:40.867514+0000","last_undegraded":"2024-04-23T01:45:40.867514+0000","last_fullsized":"2024-04-23T01:45:40.867514+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"21'327","reported_seq":350,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:51.375829+0000","last_change":"2024-04-23T01:45:16.891787+0000","last_active":"2024-04-23T01:45:51.375829+0000","last_peered":"2024-04-23T01:45:51.375829+0000","last_clean":"2024-04-23T01:45:51.375829+0000","last_became_active":"2024-04-23T01:45:16.885404+0000","last_became_peered":"2024-04-23T01:45:16.885404+0000","last_unstale":"2024-04-23T01:45:51.375829+0000","last_undegraded":"2024-04-23T01:45:51.375829+0000","last_fullsized":"2024-04-23T01:45:51.375829+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2686976,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":328,"num_write_kb":2624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:40.842025+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:45:40.842025+0000","last_peered":"2024-04-23T01:45:40.842025+0000","last_clean":"2024-04-23T01:45:40.842025+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:45:40.842025+0000","last_undegraded":"2024-04-23T01:45:40.842025+0000","last_fullsized":"2024-04-23T01:45:40.842025+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"21'279","reported_seq":302,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:51.051955+0000","last_change":"2024-04-23T01:45:16.919259+0000","last_active":"2024-04-23T01:45:51.051955+0000","last_peered":"2024-04-23T01:45:51.051955+0000","last_clean":"2024-04-23T01:45:51.051955+0000","last_became_active":"2024-04-23T01:45:16.912983+0000","last_became_peered":"2024-04-23T01:45:16.912983+0000","last_unstale":"2024-04-23T01:45:51.051955+0000","last_undegraded":"2024-04-23T01:45:51.051955+0000","last_fullsized":"2024-04-23T01:45:51.051955+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2293760,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":280,"num_write_kb":2240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:40.855832+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:45:40.855832+0000","last_peered":"2024-04-23T01:45:40.855832+0000","last_clean":"2024-04-23T01:45:40.855832+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:45:40.855832+0000","last_undegraded":"2024-04-23T01:45:40.855832+0000","last_fullsized":"2024-04-23T01:45:40.855832+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"21'367","reported_seq":390,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:51.535933+0000","last_change":"2024-04-23T01:45:16.897089+0000","last_active":"2024-04-23T01:45:51.535933+0000","last_peered":"2024-04-23T01:45:51.535933+0000","last_clean":"2024-04-23T01:45:51.535933+0000","last_became_active":"2024-04-23T01:45:16.891776+0000","last_became_peered":"2024-04-23T01:45:16.891776+0000","last_unstale":"2024-04-23T01:45:51.535933+0000","last_undegraded":"2024-04-23T01:45:51.535933+0000","last_fullsized":"2024-04-23T01:45:51.535933+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3014656,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":368,"num_write_kb":2944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:40.840035+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:45:40.840035+0000","last_peered":"2024-04-23T01:45:40.840035+0000","last_clean":"2024-04-23T01:45:40.840035+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:45:40.840035+0000","last_undegraded":"2024-04-23T01:45:40.840035+0000","last_fullsized":"2024-04-23T01:45:40.840035+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"21'323","reported_seq":348,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:51.546837+0000","last_change":"2024-04-23T01:45:16.823909+0000","last_active":"2024-04-23T01:45:51.546837+0000","last_peered":"2024-04-23T01:45:51.546837+0000","last_clean":"2024-04-23T01:45:51.546837+0000","last_became_active":"2024-04-23T01:45:16.815467+0000","last_became_peered":"2024-04-23T01:45:16.815467+0000","last_unstale":"2024-04-23T01:45:51.546837+0000","last_undegraded":"2024-04-23T01:45:51.546837+0000","last_fullsized":"2024-04-23T01:45:51.546837+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2654208,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":324,"num_write_kb":2592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"21'365","reported_seq":388,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:51.538363+0000","last_change":"2024-04-23T01:45:16.859980+0000","last_active":"2024-04-23T01:45:51.538363+0000","last_peered":"2024-04-23T01:45:51.538363+0000","last_clean":"2024-04-23T01:45:51.538363+0000","last_became_active":"2024-04-23T01:45:16.855622+0000","last_became_peered":"2024-04-23T01:45:16.855622+0000","last_unstale":"2024-04-23T01:45:51.538363+0000","last_undegraded":"2024-04-23T01:45:51.538363+0000","last_fullsized":"2024-04-23T01:45:51.538363+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2998272,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":366,"num_write_kb":2928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:35.774643+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:45:35.774643+0000","last_peered":"2024-04-23T01:45:35.774643+0000","last_clean":"2024-04-23T01:45:35.774643+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:45:35.774643+0000","last_undegraded":"2024-04-23T01:45:35.774643+0000","last_fullsized":"2024-04-23T01:45:35.774643+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"21'263","reported_seq":288,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:49.742786+0000","last_change":"2024-04-23T01:45:16.854641+0000","last_active":"2024-04-23T01:45:49.742786+0000","last_peered":"2024-04-23T01:45:49.742786+0000","last_clean":"2024-04-23T01:45:49.742786+0000","last_became_active":"2024-04-23T01:45:16.845783+0000","last_became_peered":"2024-04-23T01:45:16.845783+0000","last_unstale":"2024-04-23T01:45:49.742786+0000","last_undegraded":"2024-04-23T01:45:49.742786+0000","last_fullsized":"2024-04-23T01:45:49.742786+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162688,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":264,"num_write_kb":2112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":31,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:40.725302+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:45:40.725302+0000","last_peered":"2024-04-23T01:45:40.725302+0000","last_clean":"2024-04-23T01:45:40.725302+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:45:40.725302+0000","last_undegraded":"2024-04-23T01:45:40.725302+0000","last_fullsized":"2024-04-23T01:45:40.725302+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"21'423","reported_seq":446,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:50.096770+0000","last_change":"2024-04-23T01:45:16.845162+0000","last_active":"2024-04-23T01:45:50.096770+0000","last_peered":"2024-04-23T01:45:50.096770+0000","last_clean":"2024-04-23T01:45:50.096770+0000","last_became_active":"2024-04-23T01:45:16.829836+0000","last_became_peered":"2024-04-23T01:45:16.829836+0000","last_unstale":"2024-04-23T01:45:50.096770+0000","last_undegraded":"2024-04-23T01:45:50.096770+0000","last_fullsized":"2024-04-23T01:45:50.096770+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3473408,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":424,"num_write_kb":3392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":31,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:35.776533+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:45:35.776533+0000","last_peered":"2024-04-23T01:45:35.776533+0000","last_clean":"2024-04-23T01:45:35.776533+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:45:35.776533+0000","last_undegraded":"2024-04-23T01:45:35.776533+0000","last_fullsized":"2024-04-23T01:45:35.776533+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"21'375","reported_seq":400,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:50.587886+0000","last_change":"2024-04-23T01:45:16.885391+0000","last_active":"2024-04-23T01:45:50.587886+0000","last_peered":"2024-04-23T01:45:50.587886+0000","last_clean":"2024-04-23T01:45:50.587886+0000","last_became_active":"2024-04-23T01:45:16.880317+0000","last_became_peered":"2024-04-23T01:45:16.880317+0000","last_unstale":"2024-04-23T01:45:50.587886+0000","last_undegraded":"2024-04-23T01:45:50.587886+0000","last_fullsized":"2024-04-23T01:45:50.587886+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":3008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:40.858645+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:45:40.858645+0000","last_peered":"2024-04-23T01:45:40.858645+0000","last_clean":"2024-04-23T01:45:40.858645+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:45:40.858645+0000","last_undegraded":"2024-04-23T01:45:40.858645+0000","last_fullsized":"2024-04-23T01:45:40.858645+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":44072960,"num_objects":674,"num_object_clones":0,"num_object_copies":2022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5380,"num_write_kb":43040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":66105344,"data_stored":132210688,"data_compressed":6823467,"data_compressed_allocated":66105344,"data_compressed_original":132210688,"omap_allocated":0,"internal_metadata":0},"log_size":5364,"ondisk_log_size":5364,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705708,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":45168,"kb_used_data":18344,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94326672,"statfs":{"total":96636764160,"available":96590512128,"internally_reserved":0,"allocated":18784256,"data_stored":37737945,"data_compressed":1944379,"data_compressed_allocated":18587648,"data_compressed_original":37666816,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705708,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54052,"kb_used_data":14748,"kb_used_omap":0,"kb_used_meta":39296,"kb_avail":94317788,"statfs":{"total":96636764160,"available":96581414912,"internally_reserved":0,"allocated":15101952,"data_stored":30373337,"data_compressed":1564290,"data_compressed_allocated":14905344,"data_compressed_original":30302208,"omap_allocated":0,"internal_metadata":40239104},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705708,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":43320,"kb_used_data":16496,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94328520,"statfs":{"total":96636764160,"available":96592404480,"internally_reserved":0,"allocated":16891904,"data_stored":33485753,"data_compressed":1725345,"data_compressed_allocated":16715776,"data_compressed_original":33431552,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705708,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":53364,"kb_used_data":14060,"kb_used_omap":0,"kb_used_meta":39296,"kb_avail":94318476,"statfs":{"total":96636764160,"available":96582119424,"internally_reserved":0,"allocated":14397440,"data_stored":28964313,"data_compressed":1491500,"data_compressed_allocated":14200832,"data_compressed_original":28893184,"omap_allocated":0,"internal_metadata":40239104},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14630912,"data_stored":29261824,"data_compressed":1510192,"data_compressed_allocated":14630912,"data_compressed_original":29261824,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":17166336,"data_stored":34332672,"data_compressed":1771875,"data_compressed_allocated":17166336,"data_compressed_original":34332672,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":15290368,"data_stored":30580736,"data_compressed":1578329,"data_compressed_allocated":15290368,"data_compressed_original":30580736,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19017728,"data_stored":38035456,"data_compressed":1963071,"data_compressed_allocated":19017728,"data_compressed_original":38035456,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:45:54.507 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:45:54.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:54.637+0000 7efc0c206640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:54.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:54.645+0000 7efc0c206640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:54.792 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 37 16 5875 5859 1.23701 1.23438 0.158723 0.100843 2024-04-23T01:45:55.792 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 38 16 6036 6020 1.23756 1.25781 0.111083 0.100833 2024-04-23T01:45:56.792 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 39 16 6206 6190 1.23988 1.32812 0.0226633 0.100668 2024-04-23T01:45:56.803 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:45:56.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:56.943+0000 7f51c726d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:56.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:56.950+0000 7f51c726d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:57.792 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:45:57.794933+0000 min lat: 0.0153353 max lat: 0.332121 avg lat: 0.100942 2024-04-23T01:45:57.793 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:45:57.793 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 40 16 6346 6330 1.23622 1.09375 0.0727953 0.100942 2024-04-23T01:45:58.794 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 41 16 6498 6482 1.23503 1.1875 0.133806 0.101028 2024-04-23T01:45:58.857 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:45:58.858 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:45:58.858 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:45:58.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:58.985+0000 7f620235d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:58.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:58.994+0000 7f620235d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:59.125 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:45:59.247 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:45:59.247 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:45:59.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:59.262+0000 7ff3bfa6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:59.264 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-04-23T01:45:57.557596+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":51028512,"num_objects":773,"num_object_clones":0,"num_object_copies":2319,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":6294,"num_write_kb":51150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6238,"ondisk_log_size":6238,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":184288,"kb_used_data":72828,"kb_used_omap":0,"kb_used_meta":111424,"kb_avail":377303072,"statfs":{"total":386547056640,"available":386358345728,"internally_reserved":0,"allocated":74575872,"data_stored":149361988,"data_compressed":7696299,"data_compressed_allocated":73809920,"data_compressed_original":149094400,"omap_allocated":0,"internal_metadata":114098176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19390464,"num_objects":296,"num_object_clones":0,"num_object_copies":888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2367,"num_write_kb":18936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002278"},"pg_stats":[{"pgid":"3.f","version":"21'311","reported_seq":334,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:54.817523+0000","last_change":"2024-04-23T01:45:16.856566+0000","last_active":"2024-04-23T01:45:54.817523+0000","last_peered":"2024-04-23T01:45:54.817523+0000","last_clean":"2024-04-23T01:45:54.817523+0000","last_became_active":"2024-04-23T01:45:16.848194+0000","last_became_peered":"2024-04-23T01:45:16.848194+0000","last_unstale":"2024-04-23T01:45:54.817523+0000","last_undegraded":"2024-04-23T01:45:54.817523+0000","last_fullsized":"2024-04-23T01:45:54.817523+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2555904,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":2496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"21'415","reported_seq":440,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:56.285342+0000","last_change":"2024-04-23T01:45:16.906007+0000","last_active":"2024-04-23T01:45:56.285342+0000","last_peered":"2024-04-23T01:45:56.285342+0000","last_clean":"2024-04-23T01:45:56.285342+0000","last_became_active":"2024-04-23T01:45:16.901215+0000","last_became_peered":"2024-04-23T01:45:16.901215+0000","last_unstale":"2024-04-23T01:45:56.285342+0000","last_undegraded":"2024-04-23T01:45:56.285342+0000","last_fullsized":"2024-04-23T01:45:56.285342+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3407872,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":3328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"21'391","reported_seq":414,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:56.219042+0000","last_change":"2024-04-23T01:45:16.886628+0000","last_active":"2024-04-23T01:45:56.219042+0000","last_peered":"2024-04-23T01:45:56.219042+0000","last_clean":"2024-04-23T01:45:56.219042+0000","last_became_active":"2024-04-23T01:45:16.880681+0000","last_became_peered":"2024-04-23T01:45:16.880681+0000","last_unstale":"2024-04-23T01:45:56.219042+0000","last_undegraded":"2024-04-23T01:45:56.219042+0000","last_fullsized":"2024-04-23T01:45:56.219042+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3211264,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":392,"num_write_kb":3136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"21'399","reported_seq":422,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:56.384619+0000","last_change":"2024-04-23T01:45:16.836330+0000","last_active":"2024-04-23T01:45:56.384619+0000","last_peered":"2024-04-23T01:45:56.384619+0000","last_clean":"2024-04-23T01:45:56.384619+0000","last_became_active":"2024-04-23T01:45:16.821451+0000","last_became_peered":"2024-04-23T01:45:16.821451+0000","last_unstale":"2024-04-23T01:45:56.384619+0000","last_undegraded":"2024-04-23T01:45:56.384619+0000","last_fullsized":"2024-04-23T01:45:56.384619+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3276800,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":400,"num_write_kb":3200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"21'303","reported_seq":326,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:56.256705+0000","last_change":"2024-04-23T01:45:16.860867+0000","last_active":"2024-04-23T01:45:56.256705+0000","last_peered":"2024-04-23T01:45:56.256705+0000","last_clean":"2024-04-23T01:45:56.256705+0000","last_became_active":"2024-04-23T01:45:16.852271+0000","last_became_peered":"2024-04-23T01:45:16.852271+0000","last_unstale":"2024-04-23T01:45:56.256705+0000","last_undegraded":"2024-04-23T01:45:56.256705+0000","last_fullsized":"2024-04-23T01:45:56.256705+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490368,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"21'391","reported_seq":414,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:55.535117+0000","last_change":"2024-04-23T01:45:16.805452+0000","last_active":"2024-04-23T01:45:55.535117+0000","last_peered":"2024-04-23T01:45:55.535117+0000","last_clean":"2024-04-23T01:45:55.535117+0000","last_became_active":"2024-04-23T01:45:16.797805+0000","last_became_peered":"2024-04-23T01:45:16.797805+0000","last_unstale":"2024-04-23T01:45:55.535117+0000","last_undegraded":"2024-04-23T01:45:55.535117+0000","last_fullsized":"2024-04-23T01:45:55.535117+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3211264,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":392,"num_write_kb":3136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"21'429","reported_seq":452,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:56.556080+0000","last_change":"2024-04-23T01:45:16.893894+0000","last_active":"2024-04-23T01:45:56.556080+0000","last_peered":"2024-04-23T01:45:56.556080+0000","last_clean":"2024-04-23T01:45:56.556080+0000","last_became_active":"2024-04-23T01:45:16.888673+0000","last_became_peered":"2024-04-23T01:45:16.888673+0000","last_unstale":"2024-04-23T01:45:56.556080+0000","last_undegraded":"2024-04-23T01:45:56.556080+0000","last_fullsized":"2024-04-23T01:45:56.556080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3522560,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":430,"num_write_kb":3440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"21'394","reported_seq":417,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:56.545207+0000","last_change":"2024-04-23T01:45:16.889162+0000","last_active":"2024-04-23T01:45:56.545207+0000","last_peered":"2024-04-23T01:45:56.545207+0000","last_clean":"2024-04-23T01:45:56.545207+0000","last_became_active":"2024-04-23T01:45:16.880761+0000","last_became_peered":"2024-04-23T01:45:16.880761+0000","last_unstale":"2024-04-23T01:45:56.545207+0000","last_undegraded":"2024-04-23T01:45:56.545207+0000","last_fullsized":"2024-04-23T01:45:56.545207+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3235840,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":395,"num_write_kb":3160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":164,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:40.713962+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:45:40.713962+0000","last_peered":"2024-04-23T01:45:40.713962+0000","last_clean":"2024-04-23T01:45:40.713962+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:45:40.713962+0000","last_undegraded":"2024-04-23T01:45:40.713962+0000","last_fullsized":"2024-04-23T01:45:40.713962+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:40.867514+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:45:40.867514+0000","last_peered":"2024-04-23T01:45:40.867514+0000","last_clean":"2024-04-23T01:45:40.867514+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:45:40.867514+0000","last_undegraded":"2024-04-23T01:45:40.867514+0000","last_fullsized":"2024-04-23T01:45:40.867514+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"21'367","reported_seq":390,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:56.356673+0000","last_change":"2024-04-23T01:45:16.891787+0000","last_active":"2024-04-23T01:45:56.356673+0000","last_peered":"2024-04-23T01:45:56.356673+0000","last_clean":"2024-04-23T01:45:56.356673+0000","last_became_active":"2024-04-23T01:45:16.885404+0000","last_became_peered":"2024-04-23T01:45:16.885404+0000","last_unstale":"2024-04-23T01:45:56.356673+0000","last_undegraded":"2024-04-23T01:45:56.356673+0000","last_fullsized":"2024-04-23T01:45:56.356673+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3014656,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":368,"num_write_kb":2944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:40.842025+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:45:40.842025+0000","last_peered":"2024-04-23T01:45:40.842025+0000","last_clean":"2024-04-23T01:45:40.842025+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:45:40.842025+0000","last_undegraded":"2024-04-23T01:45:40.842025+0000","last_fullsized":"2024-04-23T01:45:40.842025+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"21'351","reported_seq":374,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:56.444655+0000","last_change":"2024-04-23T01:45:16.919259+0000","last_active":"2024-04-23T01:45:56.444655+0000","last_peered":"2024-04-23T01:45:56.444655+0000","last_clean":"2024-04-23T01:45:56.444655+0000","last_became_active":"2024-04-23T01:45:16.912983+0000","last_became_peered":"2024-04-23T01:45:16.912983+0000","last_unstale":"2024-04-23T01:45:56.444655+0000","last_undegraded":"2024-04-23T01:45:56.444655+0000","last_fullsized":"2024-04-23T01:45:56.444655+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883584,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":352,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:40.855832+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:45:40.855832+0000","last_peered":"2024-04-23T01:45:40.855832+0000","last_clean":"2024-04-23T01:45:40.855832+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:45:40.855832+0000","last_undegraded":"2024-04-23T01:45:40.855832+0000","last_fullsized":"2024-04-23T01:45:40.855832+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"21'407","reported_seq":430,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:56.034214+0000","last_change":"2024-04-23T01:45:16.897089+0000","last_active":"2024-04-23T01:45:56.034214+0000","last_peered":"2024-04-23T01:45:56.034214+0000","last_clean":"2024-04-23T01:45:56.034214+0000","last_became_active":"2024-04-23T01:45:16.891776+0000","last_became_peered":"2024-04-23T01:45:16.891776+0000","last_unstale":"2024-04-23T01:45:56.034214+0000","last_undegraded":"2024-04-23T01:45:56.034214+0000","last_fullsized":"2024-04-23T01:45:56.034214+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342336,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":408,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:40.840035+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:45:40.840035+0000","last_peered":"2024-04-23T01:45:40.840035+0000","last_clean":"2024-04-23T01:45:40.840035+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:45:40.840035+0000","last_undegraded":"2024-04-23T01:45:40.840035+0000","last_fullsized":"2024-04-23T01:45:40.840035+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"21'383","reported_seq":408,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:55.949683+0000","last_change":"2024-04-23T01:45:16.823909+0000","last_active":"2024-04-23T01:45:55.949683+0000","last_peered":"2024-04-23T01:45:55.949683+0000","last_clean":"2024-04-23T01:45:55.949683+0000","last_became_active":"2024-04-23T01:45:16.815467+0000","last_became_peered":"2024-04-23T01:45:16.815467+0000","last_unstale":"2024-04-23T01:45:55.949683+0000","last_undegraded":"2024-04-23T01:45:55.949683+0000","last_fullsized":"2024-04-23T01:45:55.949683+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":384,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"21'383","reported_seq":406,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:55.630092+0000","last_change":"2024-04-23T01:45:16.859980+0000","last_active":"2024-04-23T01:45:55.630092+0000","last_peered":"2024-04-23T01:45:55.630092+0000","last_clean":"2024-04-23T01:45:55.630092+0000","last_became_active":"2024-04-23T01:45:16.855622+0000","last_became_peered":"2024-04-23T01:45:16.855622+0000","last_unstale":"2024-04-23T01:45:55.630092+0000","last_undegraded":"2024-04-23T01:45:55.630092+0000","last_fullsized":"2024-04-23T01:45:55.630092+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":384,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:35.774643+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:45:35.774643+0000","last_peered":"2024-04-23T01:45:35.774643+0000","last_clean":"2024-04-23T01:45:35.774643+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:45:35.774643+0000","last_undegraded":"2024-04-23T01:45:35.774643+0000","last_fullsized":"2024-04-23T01:45:35.774643+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"21'311","reported_seq":336,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:55.844041+0000","last_change":"2024-04-23T01:45:16.854641+0000","last_active":"2024-04-23T01:45:55.844041+0000","last_peered":"2024-04-23T01:45:55.844041+0000","last_clean":"2024-04-23T01:45:55.844041+0000","last_became_active":"2024-04-23T01:45:16.845783+0000","last_became_peered":"2024-04-23T01:45:16.845783+0000","last_unstale":"2024-04-23T01:45:55.844041+0000","last_undegraded":"2024-04-23T01:45:55.844041+0000","last_fullsized":"2024-04-23T01:45:55.844041+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2555904,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":2496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":31,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:40.725302+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:45:40.725302+0000","last_peered":"2024-04-23T01:45:40.725302+0000","last_clean":"2024-04-23T01:45:40.725302+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:45:40.725302+0000","last_undegraded":"2024-04-23T01:45:40.725302+0000","last_fullsized":"2024-04-23T01:45:40.725302+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"21'487","reported_seq":510,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:56.065743+0000","last_change":"2024-04-23T01:45:16.845162+0000","last_active":"2024-04-23T01:45:56.065743+0000","last_peered":"2024-04-23T01:45:56.065743+0000","last_clean":"2024-04-23T01:45:56.065743+0000","last_became_active":"2024-04-23T01:45:16.829836+0000","last_became_peered":"2024-04-23T01:45:16.829836+0000","last_unstale":"2024-04-23T01:45:56.065743+0000","last_undegraded":"2024-04-23T01:45:56.065743+0000","last_fullsized":"2024-04-23T01:45:56.065743+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3997696,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":488,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":31,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:35.776533+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:45:35.776533+0000","last_peered":"2024-04-23T01:45:35.776533+0000","last_clean":"2024-04-23T01:45:35.776533+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:45:35.776533+0000","last_undegraded":"2024-04-23T01:45:35.776533+0000","last_fullsized":"2024-04-23T01:45:35.776533+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"21'419","reported_seq":444,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:56.570879+0000","last_change":"2024-04-23T01:45:16.885391+0000","last_active":"2024-04-23T01:45:56.570879+0000","last_peered":"2024-04-23T01:45:56.570879+0000","last_clean":"2024-04-23T01:45:56.570879+0000","last_became_active":"2024-04-23T01:45:16.880317+0000","last_became_peered":"2024-04-23T01:45:16.880317+0000","last_unstale":"2024-04-23T01:45:56.570879+0000","last_undegraded":"2024-04-23T01:45:56.570879+0000","last_fullsized":"2024-04-23T01:45:56.570879+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3440640,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":420,"num_write_kb":3360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:40.858645+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:45:40.858645+0000","last_peered":"2024-04-23T01:45:40.858645+0000","last_clean":"2024-04-23T01:45:40.858645+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:45:40.858645+0000","last_undegraded":"2024-04-23T01:45:40.858645+0000","last_fullsized":"2024-04-23T01:45:40.858645+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":50438144,"num_objects":771,"num_object_clones":0,"num_object_copies":2313,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6157,"num_write_kb":49256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":75649024,"data_stored":151298048,"data_compressed":7809057,"data_compressed_allocated":75649024,"data_compressed_original":151298048,"omap_allocated":0,"internal_metadata":0},"log_size":6141,"ondisk_log_size":6141,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705713,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51976,"kb_used_data":20992,"kb_used_omap":0,"kb_used_meta":30976,"kb_avail":94319864,"statfs":{"total":96636764160,"available":96583540736,"internally_reserved":0,"allocated":21495808,"data_stored":43161049,"data_compressed":2224405,"data_compressed_allocated":21299200,"data_compressed_original":43089920,"omap_allocated":0,"internal_metadata":31719424},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705713,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":43588,"kb_used_data":16764,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94328252,"statfs":{"total":96636764160,"available":96592130048,"internally_reserved":0,"allocated":17166336,"data_stored":34502105,"data_compressed":1777482,"data_compressed_allocated":16969728,"data_compressed_original":34430976,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705713,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":45544,"kb_used_data":18720,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94326296,"statfs":{"total":96636764160,"available":96590127104,"internally_reserved":0,"allocated":19169280,"data_stored":38040505,"data_compressed":1960533,"data_compressed_allocated":18993152,"data_compressed_original":37986304,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705713,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":43180,"kb_used_data":16352,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94328660,"statfs":{"total":96636764160,"available":96592547840,"internally_reserved":0,"allocated":16744448,"data_stored":33658329,"data_compressed":1733879,"data_compressed_allocated":16547840,"data_compressed_original":33587200,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16932864,"data_stored":33865728,"data_compressed":1747918,"data_compressed_allocated":16932864,"data_compressed_original":33865728,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":19480576,"data_stored":38961152,"data_compressed":2010870,"data_compressed_allocated":19480576,"data_compressed_original":38961152,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":17387520,"data_stored":34775040,"data_compressed":1794905,"data_compressed_allocated":17387520,"data_compressed_original":34775040,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":21848064,"data_stored":43696128,"data_compressed":2255364,"data_compressed_allocated":21848064,"data_compressed_original":43696128,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:45:59.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:45:59.271+0000 7ff3bfa6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:45:59.793 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 42 16 6640 6624 1.23204 1.10938 0.054721 0.101282 2024-04-23T01:46:00.793 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 43 16 6795 6779 1.23154 1.21094 0.107896 0.101381 2024-04-23T01:46:01.436 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:46:01.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:01.569+0000 7f3aed39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:01.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:01.576+0000 7f3aed39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:01.793 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 44 16 6942 6926 1.22965 1.14844 0.172126 0.101395 2024-04-23T01:46:02.793 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 45 16 7088 7072 1.22767 1.14062 0.100419 0.101677 2024-04-23T01:46:03.761 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:46:03.793 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 46 16 7239 7223 1.22663 1.17969 0.123871 0.101765 2024-04-23T01:46:03.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:03.911+0000 7f74400ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:03.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:03.918+0000 7f74400ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:04.266 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:46:04.266 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:46:04.266 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:46:04.267 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:46:04.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:04.401+0000 7f75f399d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:04.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:04.415+0000 7f75f399d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:04.451 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:46:04.451 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:46:04.451 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:46:04.451 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:46:04.452 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:46:04.452 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:46:04.452 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:46:04.452 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:46:04.452 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:46:04.452 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:46:04.452 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:46:04.452 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:46:04.455 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:46:04.455 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:46:04.456 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:46:04.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:04.580+0000 7f1ee4c2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:04.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:04.591+0000 7f1ee4c2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:04.606 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:46:04.606 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:46:04.606 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:46:04.606 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:46:04.606 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:46:04.606 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:46:04.606 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:46:04.606 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:46:04.606 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:46:04.606 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:46:04.606 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:46:04.607 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:46:04.609 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:46:04.609 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:46:04.609 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:46:04.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:04.737+0000 7f9d0df9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:04.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:04.748+0000 7f9d0df9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:04.773 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:46:04.773 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:46:04.773 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:46:04.773 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:46:04.773 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:46:04.773 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:46:04.773 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:46:04.773 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:46:04.774 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:46:04.774 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:46:04.774 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:46:04.774 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:46:04.776 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:46:04.777 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:46:04.777 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:46:04.793 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 47 16 7397 7381 1.22679 1.23438 0.0292525 0.101764 2024-04-23T01:46:04.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:04.913+0000 7fcaeca6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:04.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:04.927+0000 7fcaeca6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:04.955 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:46:04.955 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:46:04.955 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:46:04.955 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:46:04.955 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:46:04.955 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:46:04.955 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:46:04.955 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:46:04.956 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:46:04.956 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:46:04.956 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:46:04.956 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:46:04.958 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:46:04.958 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:46:05.793 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 48 16 7557 7541 1.22727 1.25 0.0242599 0.101705 2024-04-23T01:46:06.073 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:46:06.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:06.214+0000 7f952d15d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:06.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:06.221+0000 7f952d15d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:06.793 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 49 16 7712 7696 1.22694 1.21094 0.126057 0.10175 2024-04-23T01:46:07.793 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 50 16 7865 7849 1.2263 1.19531 0.11528 0.101795 2024-04-23T01:46:08.343 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:46:08.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:08.470+0000 7fba2ec64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:08.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:08.477+0000 7fba2ec64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:08.793 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 51 16 8009 7993 1.22431 1.125 0.151076 0.101955 2024-04-23T01:46:09.794 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 52 16 8151 8135 1.2221 1.10938 0.151319 0.102146 2024-04-23T01:46:09.959 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:46:09.959 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-23T01:46:10.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:10.079+0000 7f2ba30ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:10.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:10.091+0000 7f2ba30ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:10.651 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:46:10.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:10.776+0000 7fa7f175d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:10.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:10.783+0000 7fa7f175d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:10.793 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 53 16 8279 8263 1.21791 1 0.159548 0.102517 2024-04-23T01:46:11.185 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.0 to 1 (10000) 2024-04-23T01:46:11.202 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-23T01:46:11.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:11.326+0000 7eff2f66d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:11.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:11.338+0000 7eff2f66d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:11.794 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 54 16 8429 8413 1.21705 1.17188 0.114619 0.102609 2024-04-23T01:46:12.198 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.1 to 1 (10000) 2024-04-23T01:46:12.214 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-23T01:46:12.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:12.353+0000 7f206bead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:12.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:12.366+0000 7f206bead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:12.794 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 55 16 8575 8559 1.21566 1.14062 0.201293 0.102659 2024-04-23T01:46:12.985 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:46:13.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:13.108+0000 7fccb7e2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:13.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:13.114+0000 7fccb7e2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:13.199 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.2 to 1 (10000) 2024-04-23T01:46:13.218 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-23T01:46:13.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:13.352+0000 7fe405aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:13.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:13.369+0000 7fe405aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:13.794 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 56 16 8700 8684 1.21139 0.976562 0.199151 0.103025 2024-04-23T01:46:14.206 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.3 to 1 (10000) 2024-04-23T01:46:14.229 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-23T01:46:14.230 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:46:14.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:14.387+0000 7f0d0571d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:14.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:14.399+0000 7f0d0571d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:14.678 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:46:14.679 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:46:14.698 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2024-04-23T01:46:14.209349+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":69288480,"num_objects":1052,"num_object_clones":0,"num_object_copies":3156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":8523,"num_write_kb":68982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8472,"ondisk_log_size":8472,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":240328,"kb_used_data":99752,"kb_used_omap":0,"kb_used_meta":140544,"kb_avail":377247032,"statfs":{"total":386547056640,"available":386300960768,"internally_reserved":0,"allocated":102146048,"data_stored":204502340,"data_compressed":10543512,"data_compressed_allocated":101380096,"data_compressed_original":204234752,"omap_allocated":0,"internal_metadata":143917056},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5906432,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":721,"num_write_kb":5768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.649382"},"pg_stats":[{"pgid":"3.f","version":"21'431","reported_seq":454,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.554055+0000","last_change":"2024-04-23T01:45:16.856566+0000","last_active":"2024-04-23T01:46:11.554055+0000","last_peered":"2024-04-23T01:46:11.554055+0000","last_clean":"2024-04-23T01:46:11.554055+0000","last_became_active":"2024-04-23T01:45:16.848194+0000","last_became_peered":"2024-04-23T01:45:16.848194+0000","last_unstale":"2024-04-23T01:46:11.554055+0000","last_undegraded":"2024-04-23T01:46:11.554055+0000","last_fullsized":"2024-04-23T01:46:11.554055+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3538944,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":432,"num_write_kb":3456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"21'553","reported_seq":578,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.545952+0000","last_change":"2024-04-23T01:45:16.906007+0000","last_active":"2024-04-23T01:46:11.545952+0000","last_peered":"2024-04-23T01:46:11.545952+0000","last_clean":"2024-04-23T01:46:11.545952+0000","last_became_active":"2024-04-23T01:45:16.901215+0000","last_became_peered":"2024-04-23T01:45:16.901215+0000","last_unstale":"2024-04-23T01:46:11.545952+0000","last_undegraded":"2024-04-23T01:46:11.545952+0000","last_fullsized":"2024-04-23T01:46:11.545952+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4538368,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":554,"num_write_kb":4432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"21'520","reported_seq":545,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.598508+0000","last_change":"2024-04-23T01:45:16.886628+0000","last_active":"2024-04-23T01:46:11.598508+0000","last_peered":"2024-04-23T01:46:11.598508+0000","last_clean":"2024-04-23T01:46:11.598508+0000","last_became_active":"2024-04-23T01:45:16.880681+0000","last_became_peered":"2024-04-23T01:45:16.880681+0000","last_unstale":"2024-04-23T01:46:11.598508+0000","last_undegraded":"2024-04-23T01:46:11.598508+0000","last_fullsized":"2024-04-23T01:46:11.598508+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259840,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":520,"num_write_kb":4160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"21'543","reported_seq":566,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.309019+0000","last_change":"2024-04-23T01:45:16.836330+0000","last_active":"2024-04-23T01:46:11.309019+0000","last_peered":"2024-04-23T01:46:11.309019+0000","last_clean":"2024-04-23T01:46:11.309019+0000","last_became_active":"2024-04-23T01:45:16.821451+0000","last_became_peered":"2024-04-23T01:45:16.821451+0000","last_unstale":"2024-04-23T01:46:11.309019+0000","last_undegraded":"2024-04-23T01:46:11.309019+0000","last_fullsized":"2024-04-23T01:46:11.309019+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":544,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'487","reported_seq":513,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.280821+0000","last_change":"2024-04-23T01:45:16.860867+0000","last_active":"2024-04-23T01:46:11.280821+0000","last_peered":"2024-04-23T01:46:11.280821+0000","last_clean":"2024-04-23T01:46:11.280821+0000","last_became_active":"2024-04-23T01:45:16.852271+0000","last_became_peered":"2024-04-23T01:45:16.852271+0000","last_unstale":"2024-04-23T01:46:11.280821+0000","last_undegraded":"2024-04-23T01:46:11.280821+0000","last_fullsized":"2024-04-23T01:46:11.280821+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3997696,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":488,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"21'527","reported_seq":550,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.454188+0000","last_change":"2024-04-23T01:45:16.805452+0000","last_active":"2024-04-23T01:46:11.454188+0000","last_peered":"2024-04-23T01:46:11.454188+0000","last_clean":"2024-04-23T01:46:11.454188+0000","last_became_active":"2024-04-23T01:45:16.797805+0000","last_became_peered":"2024-04-23T01:45:16.797805+0000","last_unstale":"2024-04-23T01:46:11.454188+0000","last_undegraded":"2024-04-23T01:46:11.454188+0000","last_fullsized":"2024-04-23T01:46:11.454188+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":528,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"21'591","reported_seq":614,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.247486+0000","last_change":"2024-04-23T01:45:16.893894+0000","last_active":"2024-04-23T01:46:11.247486+0000","last_peered":"2024-04-23T01:46:11.247486+0000","last_clean":"2024-04-23T01:46:11.247486+0000","last_became_active":"2024-04-23T01:45:16.888673+0000","last_became_peered":"2024-04-23T01:45:16.888673+0000","last_unstale":"2024-04-23T01:46:11.247486+0000","last_undegraded":"2024-04-23T01:46:11.247486+0000","last_fullsized":"2024-04-23T01:46:11.247486+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":592,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"21'576","reported_seq":601,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.600503+0000","last_change":"2024-04-23T01:45:16.889162+0000","last_active":"2024-04-23T01:46:11.600503+0000","last_peered":"2024-04-23T01:46:11.600503+0000","last_clean":"2024-04-23T01:46:11.600503+0000","last_became_active":"2024-04-23T01:45:16.880761+0000","last_became_peered":"2024-04-23T01:45:16.880761+0000","last_unstale":"2024-04-23T01:46:11.600503+0000","last_undegraded":"2024-04-23T01:46:11.600503+0000","last_fullsized":"2024-04-23T01:46:11.600503+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718592,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":576,"num_write_kb":4608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.219703+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:46:11.219703+0000","last_peered":"2024-04-23T01:46:11.219703+0000","last_clean":"2024-04-23T01:46:11.219703+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:46:11.219703+0000","last_undegraded":"2024-04-23T01:46:11.219703+0000","last_fullsized":"2024-04-23T01:46:11.219703+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-23T01:45:40.867514+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:45:40.867514+0000","last_peered":"2024-04-23T01:45:40.867514+0000","last_clean":"2024-04-23T01:45:40.867514+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:45:40.867514+0000","last_undegraded":"2024-04-23T01:45:40.867514+0000","last_fullsized":"2024-04-23T01:45:40.867514+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"21'480","reported_seq":505,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.596231+0000","last_change":"2024-04-23T01:45:16.891787+0000","last_active":"2024-04-23T01:46:11.596231+0000","last_peered":"2024-04-23T01:46:11.596231+0000","last_clean":"2024-04-23T01:46:11.596231+0000","last_became_active":"2024-04-23T01:45:16.885404+0000","last_became_peered":"2024-04-23T01:45:16.885404+0000","last_unstale":"2024-04-23T01:46:11.596231+0000","last_undegraded":"2024-04-23T01:46:11.596231+0000","last_fullsized":"2024-04-23T01:46:11.596231+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":480,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:40.842025+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:45:40.842025+0000","last_peered":"2024-04-23T01:45:40.842025+0000","last_clean":"2024-04-23T01:45:40.842025+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:45:40.842025+0000","last_undegraded":"2024-04-23T01:45:40.842025+0000","last_fullsized":"2024-04-23T01:45:40.842025+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"22'466","reported_seq":492,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.535386+0000","last_change":"2024-04-23T01:45:16.919259+0000","last_active":"2024-04-23T01:46:11.535386+0000","last_peered":"2024-04-23T01:46:11.535386+0000","last_clean":"2024-04-23T01:46:11.535386+0000","last_became_active":"2024-04-23T01:45:16.912983+0000","last_became_peered":"2024-04-23T01:45:16.912983+0000","last_unstale":"2024-04-23T01:46:11.535386+0000","last_undegraded":"2024-04-23T01:46:11.535386+0000","last_fullsized":"2024-04-23T01:46:11.535386+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3825664,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":467,"num_write_kb":3736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.594697+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:46:11.594697+0000","last_peered":"2024-04-23T01:46:11.594697+0000","last_clean":"2024-04-23T01:46:11.594697+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:46:11.594697+0000","last_undegraded":"2024-04-23T01:46:11.594697+0000","last_fullsized":"2024-04-23T01:46:11.594697+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"21'480","reported_seq":505,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.228844+0000","last_change":"2024-04-23T01:45:16.897089+0000","last_active":"2024-04-23T01:46:11.228844+0000","last_peered":"2024-04-23T01:46:11.228844+0000","last_clean":"2024-04-23T01:46:11.228844+0000","last_became_active":"2024-04-23T01:45:16.891776+0000","last_became_peered":"2024-04-23T01:45:16.891776+0000","last_unstale":"2024-04-23T01:46:11.228844+0000","last_undegraded":"2024-04-23T01:46:11.228844+0000","last_fullsized":"2024-04-23T01:46:11.228844+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":480,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:40.840035+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:45:40.840035+0000","last_peered":"2024-04-23T01:45:40.840035+0000","last_clean":"2024-04-23T01:45:40.840035+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:45:40.840035+0000","last_undegraded":"2024-04-23T01:45:40.840035+0000","last_fullsized":"2024-04-23T01:45:40.840035+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"21'532","reported_seq":557,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.544196+0000","last_change":"2024-04-23T01:45:16.823909+0000","last_active":"2024-04-23T01:46:11.544196+0000","last_peered":"2024-04-23T01:46:11.544196+0000","last_clean":"2024-04-23T01:46:11.544196+0000","last_became_active":"2024-04-23T01:45:16.815467+0000","last_became_peered":"2024-04-23T01:45:16.815467+0000","last_unstale":"2024-04-23T01:46:11.544196+0000","last_undegraded":"2024-04-23T01:46:11.544196+0000","last_fullsized":"2024-04-23T01:46:11.544196+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4366336,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":533,"num_write_kb":4264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"21'568","reported_seq":593,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.599244+0000","last_change":"2024-04-23T01:45:16.859980+0000","last_active":"2024-04-23T01:46:11.599244+0000","last_peered":"2024-04-23T01:46:11.599244+0000","last_clean":"2024-04-23T01:46:11.599244+0000","last_became_active":"2024-04-23T01:45:16.855622+0000","last_became_peered":"2024-04-23T01:45:16.855622+0000","last_unstale":"2024-04-23T01:46:11.599244+0000","last_undegraded":"2024-04-23T01:46:11.599244+0000","last_fullsized":"2024-04-23T01:46:11.599244+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":568,"log_dups_size":0,"ondisk_log_size":568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":568,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:35.774643+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:45:35.774643+0000","last_peered":"2024-04-23T01:45:35.774643+0000","last_clean":"2024-04-23T01:45:35.774643+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:45:35.774643+0000","last_undegraded":"2024-04-23T01:45:35.774643+0000","last_fullsized":"2024-04-23T01:45:35.774643+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"21'423","reported_seq":448,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.172787+0000","last_change":"2024-04-23T01:45:16.854641+0000","last_active":"2024-04-23T01:46:11.172787+0000","last_peered":"2024-04-23T01:46:11.172787+0000","last_clean":"2024-04-23T01:46:11.172787+0000","last_became_active":"2024-04-23T01:45:16.845783+0000","last_became_peered":"2024-04-23T01:45:16.845783+0000","last_unstale":"2024-04-23T01:46:11.172787+0000","last_undegraded":"2024-04-23T01:46:11.172787+0000","last_fullsized":"2024-04-23T01:46:11.172787+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3473408,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":424,"num_write_kb":3392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":34,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.228810+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:46:11.228810+0000","last_peered":"2024-04-23T01:46:11.228810+0000","last_clean":"2024-04-23T01:46:11.228810+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:46:11.228810+0000","last_undegraded":"2024-04-23T01:46:11.228810+0000","last_fullsized":"2024-04-23T01:46:11.228810+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"22'655","reported_seq":681,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.457211+0000","last_change":"2024-04-23T01:45:16.845162+0000","last_active":"2024-04-23T01:46:11.457211+0000","last_peered":"2024-04-23T01:46:11.457211+0000","last_clean":"2024-04-23T01:46:11.457211+0000","last_became_active":"2024-04-23T01:45:16.829836+0000","last_became_peered":"2024-04-23T01:45:16.829836+0000","last_unstale":"2024-04-23T01:46:11.457211+0000","last_undegraded":"2024-04-23T01:46:11.457211+0000","last_fullsized":"2024-04-23T01:46:11.457211+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":31,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:35.776533+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:45:35.776533+0000","last_peered":"2024-04-23T01:45:35.776533+0000","last_clean":"2024-04-23T01:45:35.776533+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:45:35.776533+0000","last_undegraded":"2024-04-23T01:45:35.776533+0000","last_fullsized":"2024-04-23T01:45:35.776533+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"21'543","reported_seq":568,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.017341+0000","last_change":"2024-04-23T01:45:16.885391+0000","last_active":"2024-04-23T01:46:11.017341+0000","last_peered":"2024-04-23T01:46:11.017341+0000","last_clean":"2024-04-23T01:46:11.017341+0000","last_became_active":"2024-04-23T01:45:16.880317+0000","last_became_peered":"2024-04-23T01:45:16.880317+0000","last_unstale":"2024-04-23T01:46:11.017341+0000","last_undegraded":"2024-04-23T01:46:11.017341+0000","last_fullsized":"2024-04-23T01:46:11.017341+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":544,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":30,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-23T01:45:40.859361+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:45:40.859361+0000","last_peered":"2024-04-23T01:45:40.859361+0000","last_clean":"2024-04-23T01:45:40.859361+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:45:40.859361+0000","last_undegraded":"2024-04-23T01:45:40.859361+0000","last_fullsized":"2024-04-23T01:45:40.859361+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":68698112,"num_objects":1050,"num_object_clones":0,"num_object_copies":3150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8386,"num_write_kb":67088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":103030784,"data_stored":206061568,"data_compressed":10636812,"data_compressed_allocated":103030784,"data_compressed_original":206061568,"omap_allocated":0,"internal_metadata":0},"log_size":8375,"ondisk_log_size":8375,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705728,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":68116,"kb_used_data":28812,"kb_used_omap":0,"kb_used_meta":39296,"kb_avail":94303724,"statfs":{"total":96636764160,"available":96567013376,"internally_reserved":0,"allocated":29503488,"data_stored":59176409,"data_compressed":3051370,"data_compressed_allocated":29306880,"data_compressed_original":59105280,"omap_allocated":0,"internal_metadata":40239104},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705728,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58244,"kb_used_data":23100,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94313596,"statfs":{"total":96636764160,"available":96577122304,"internally_reserved":0,"allocated":23654400,"data_stored":47478233,"data_compressed":2447514,"data_compressed_allocated":23457792,"data_compressed_original":47407104,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705728,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60532,"kb_used_data":25388,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94311308,"statfs":{"total":96636764160,"available":96574779392,"internally_reserved":0,"allocated":25997312,"data_stored":51696569,"data_compressed":2665674,"data_compressed_allocated":25821184,"data_compressed_original":51642368,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705728,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":53436,"kb_used_data":22452,"kb_used_omap":0,"kb_used_meta":30976,"kb_avail":94318404,"statfs":{"total":96636764160,"available":96582045696,"internally_reserved":0,"allocated":22990848,"data_stored":46151129,"data_compressed":2378954,"data_compressed_allocated":22794240,"data_compressed_original":46080000,"omap_allocated":0,"internal_metadata":31719424},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":23195648,"data_stored":46391296,"data_compressed":2394685,"data_compressed_allocated":23195648,"data_compressed_original":46391296,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26198016,"data_stored":52396032,"data_compressed":2704590,"data_compressed_allocated":26198016,"data_compressed_original":52396032,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":23883776,"data_stored":47767552,"data_compressed":2465783,"data_compressed_allocated":23883776,"data_compressed_original":47767552,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":29753344,"data_stored":59506688,"data_compressed":3071754,"data_compressed_allocated":29753344,"data_compressed_original":59506688,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:46:14.700 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:46:14.794 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 57 16 8837 8821 1.20891 1.07031 0.271837 0.103188 2024-04-23T01:46:14.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:14.834+0000 7f2b3e31d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:14.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:14.846+0000 7f2b3e31d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:15.120 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:46:15.121 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:46:15.139 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2024-04-23T01:46:14.209349+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":69288480,"num_objects":1052,"num_object_clones":0,"num_object_copies":3156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":8523,"num_write_kb":68982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8472,"ondisk_log_size":8472,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":240328,"kb_used_data":99752,"kb_used_omap":0,"kb_used_meta":140544,"kb_avail":377247032,"statfs":{"total":386547056640,"available":386300960768,"internally_reserved":0,"allocated":102146048,"data_stored":204502340,"data_compressed":10543512,"data_compressed_allocated":101380096,"data_compressed_original":204234752,"omap_allocated":0,"internal_metadata":143917056},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5906432,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":721,"num_write_kb":5768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.649382"},"pg_stats":[{"pgid":"3.f","version":"21'431","reported_seq":454,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.554055+0000","last_change":"2024-04-23T01:45:16.856566+0000","last_active":"2024-04-23T01:46:11.554055+0000","last_peered":"2024-04-23T01:46:11.554055+0000","last_clean":"2024-04-23T01:46:11.554055+0000","last_became_active":"2024-04-23T01:45:16.848194+0000","last_became_peered":"2024-04-23T01:45:16.848194+0000","last_unstale":"2024-04-23T01:46:11.554055+0000","last_undegraded":"2024-04-23T01:46:11.554055+0000","last_fullsized":"2024-04-23T01:46:11.554055+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3538944,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":432,"num_write_kb":3456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"21'553","reported_seq":578,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.545952+0000","last_change":"2024-04-23T01:45:16.906007+0000","last_active":"2024-04-23T01:46:11.545952+0000","last_peered":"2024-04-23T01:46:11.545952+0000","last_clean":"2024-04-23T01:46:11.545952+0000","last_became_active":"2024-04-23T01:45:16.901215+0000","last_became_peered":"2024-04-23T01:45:16.901215+0000","last_unstale":"2024-04-23T01:46:11.545952+0000","last_undegraded":"2024-04-23T01:46:11.545952+0000","last_fullsized":"2024-04-23T01:46:11.545952+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4538368,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":554,"num_write_kb":4432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"21'520","reported_seq":545,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.598508+0000","last_change":"2024-04-23T01:45:16.886628+0000","last_active":"2024-04-23T01:46:11.598508+0000","last_peered":"2024-04-23T01:46:11.598508+0000","last_clean":"2024-04-23T01:46:11.598508+0000","last_became_active":"2024-04-23T01:45:16.880681+0000","last_became_peered":"2024-04-23T01:45:16.880681+0000","last_unstale":"2024-04-23T01:46:11.598508+0000","last_undegraded":"2024-04-23T01:46:11.598508+0000","last_fullsized":"2024-04-23T01:46:11.598508+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259840,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":520,"num_write_kb":4160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"21'543","reported_seq":566,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.309019+0000","last_change":"2024-04-23T01:45:16.836330+0000","last_active":"2024-04-23T01:46:11.309019+0000","last_peered":"2024-04-23T01:46:11.309019+0000","last_clean":"2024-04-23T01:46:11.309019+0000","last_became_active":"2024-04-23T01:45:16.821451+0000","last_became_peered":"2024-04-23T01:45:16.821451+0000","last_unstale":"2024-04-23T01:46:11.309019+0000","last_undegraded":"2024-04-23T01:46:11.309019+0000","last_fullsized":"2024-04-23T01:46:11.309019+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":544,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'487","reported_seq":513,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.280821+0000","last_change":"2024-04-23T01:45:16.860867+0000","last_active":"2024-04-23T01:46:11.280821+0000","last_peered":"2024-04-23T01:46:11.280821+0000","last_clean":"2024-04-23T01:46:11.280821+0000","last_became_active":"2024-04-23T01:45:16.852271+0000","last_became_peered":"2024-04-23T01:45:16.852271+0000","last_unstale":"2024-04-23T01:46:11.280821+0000","last_undegraded":"2024-04-23T01:46:11.280821+0000","last_fullsized":"2024-04-23T01:46:11.280821+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3997696,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":488,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"21'527","reported_seq":550,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.454188+0000","last_change":"2024-04-23T01:45:16.805452+0000","last_active":"2024-04-23T01:46:11.454188+0000","last_peered":"2024-04-23T01:46:11.454188+0000","last_clean":"2024-04-23T01:46:11.454188+0000","last_became_active":"2024-04-23T01:45:16.797805+0000","last_became_peered":"2024-04-23T01:45:16.797805+0000","last_unstale":"2024-04-23T01:46:11.454188+0000","last_undegraded":"2024-04-23T01:46:11.454188+0000","last_fullsized":"2024-04-23T01:46:11.454188+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":528,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"21'591","reported_seq":614,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.247486+0000","last_change":"2024-04-23T01:45:16.893894+0000","last_active":"2024-04-23T01:46:11.247486+0000","last_peered":"2024-04-23T01:46:11.247486+0000","last_clean":"2024-04-23T01:46:11.247486+0000","last_became_active":"2024-04-23T01:45:16.888673+0000","last_became_peered":"2024-04-23T01:45:16.888673+0000","last_unstale":"2024-04-23T01:46:11.247486+0000","last_undegraded":"2024-04-23T01:46:11.247486+0000","last_fullsized":"2024-04-23T01:46:11.247486+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":592,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"21'576","reported_seq":601,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.600503+0000","last_change":"2024-04-23T01:45:16.889162+0000","last_active":"2024-04-23T01:46:11.600503+0000","last_peered":"2024-04-23T01:46:11.600503+0000","last_clean":"2024-04-23T01:46:11.600503+0000","last_became_active":"2024-04-23T01:45:16.880761+0000","last_became_peered":"2024-04-23T01:45:16.880761+0000","last_unstale":"2024-04-23T01:46:11.600503+0000","last_undegraded":"2024-04-23T01:46:11.600503+0000","last_fullsized":"2024-04-23T01:46:11.600503+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718592,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":576,"num_write_kb":4608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.219703+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:46:11.219703+0000","last_peered":"2024-04-23T01:46:11.219703+0000","last_clean":"2024-04-23T01:46:11.219703+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:46:11.219703+0000","last_undegraded":"2024-04-23T01:46:11.219703+0000","last_fullsized":"2024-04-23T01:46:11.219703+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-23T01:45:40.867514+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:45:40.867514+0000","last_peered":"2024-04-23T01:45:40.867514+0000","last_clean":"2024-04-23T01:45:40.867514+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:45:40.867514+0000","last_undegraded":"2024-04-23T01:45:40.867514+0000","last_fullsized":"2024-04-23T01:45:40.867514+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"21'480","reported_seq":505,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.596231+0000","last_change":"2024-04-23T01:45:16.891787+0000","last_active":"2024-04-23T01:46:11.596231+0000","last_peered":"2024-04-23T01:46:11.596231+0000","last_clean":"2024-04-23T01:46:11.596231+0000","last_became_active":"2024-04-23T01:45:16.885404+0000","last_became_peered":"2024-04-23T01:45:16.885404+0000","last_unstale":"2024-04-23T01:46:11.596231+0000","last_undegraded":"2024-04-23T01:46:11.596231+0000","last_fullsized":"2024-04-23T01:46:11.596231+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":480,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:40.842025+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:45:40.842025+0000","last_peered":"2024-04-23T01:45:40.842025+0000","last_clean":"2024-04-23T01:45:40.842025+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:45:40.842025+0000","last_undegraded":"2024-04-23T01:45:40.842025+0000","last_fullsized":"2024-04-23T01:45:40.842025+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"22'466","reported_seq":492,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.535386+0000","last_change":"2024-04-23T01:45:16.919259+0000","last_active":"2024-04-23T01:46:11.535386+0000","last_peered":"2024-04-23T01:46:11.535386+0000","last_clean":"2024-04-23T01:46:11.535386+0000","last_became_active":"2024-04-23T01:45:16.912983+0000","last_became_peered":"2024-04-23T01:45:16.912983+0000","last_unstale":"2024-04-23T01:46:11.535386+0000","last_undegraded":"2024-04-23T01:46:11.535386+0000","last_fullsized":"2024-04-23T01:46:11.535386+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3825664,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":467,"num_write_kb":3736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.594697+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:46:11.594697+0000","last_peered":"2024-04-23T01:46:11.594697+0000","last_clean":"2024-04-23T01:46:11.594697+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:46:11.594697+0000","last_undegraded":"2024-04-23T01:46:11.594697+0000","last_fullsized":"2024-04-23T01:46:11.594697+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"21'480","reported_seq":505,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.228844+0000","last_change":"2024-04-23T01:45:16.897089+0000","last_active":"2024-04-23T01:46:11.228844+0000","last_peered":"2024-04-23T01:46:11.228844+0000","last_clean":"2024-04-23T01:46:11.228844+0000","last_became_active":"2024-04-23T01:45:16.891776+0000","last_became_peered":"2024-04-23T01:45:16.891776+0000","last_unstale":"2024-04-23T01:46:11.228844+0000","last_undegraded":"2024-04-23T01:46:11.228844+0000","last_fullsized":"2024-04-23T01:46:11.228844+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":480,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:40.840035+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:45:40.840035+0000","last_peered":"2024-04-23T01:45:40.840035+0000","last_clean":"2024-04-23T01:45:40.840035+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:45:40.840035+0000","last_undegraded":"2024-04-23T01:45:40.840035+0000","last_fullsized":"2024-04-23T01:45:40.840035+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"21'532","reported_seq":557,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.544196+0000","last_change":"2024-04-23T01:45:16.823909+0000","last_active":"2024-04-23T01:46:11.544196+0000","last_peered":"2024-04-23T01:46:11.544196+0000","last_clean":"2024-04-23T01:46:11.544196+0000","last_became_active":"2024-04-23T01:45:16.815467+0000","last_became_peered":"2024-04-23T01:45:16.815467+0000","last_unstale":"2024-04-23T01:46:11.544196+0000","last_undegraded":"2024-04-23T01:46:11.544196+0000","last_fullsized":"2024-04-23T01:46:11.544196+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4366336,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":533,"num_write_kb":4264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"21'568","reported_seq":593,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.599244+0000","last_change":"2024-04-23T01:45:16.859980+0000","last_active":"2024-04-23T01:46:11.599244+0000","last_peered":"2024-04-23T01:46:11.599244+0000","last_clean":"2024-04-23T01:46:11.599244+0000","last_became_active":"2024-04-23T01:45:16.855622+0000","last_became_peered":"2024-04-23T01:45:16.855622+0000","last_unstale":"2024-04-23T01:46:11.599244+0000","last_undegraded":"2024-04-23T01:46:11.599244+0000","last_fullsized":"2024-04-23T01:46:11.599244+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":568,"log_dups_size":0,"ondisk_log_size":568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":568,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:35.774643+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:45:35.774643+0000","last_peered":"2024-04-23T01:45:35.774643+0000","last_clean":"2024-04-23T01:45:35.774643+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:45:35.774643+0000","last_undegraded":"2024-04-23T01:45:35.774643+0000","last_fullsized":"2024-04-23T01:45:35.774643+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"21'423","reported_seq":448,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.172787+0000","last_change":"2024-04-23T01:45:16.854641+0000","last_active":"2024-04-23T01:46:11.172787+0000","last_peered":"2024-04-23T01:46:11.172787+0000","last_clean":"2024-04-23T01:46:11.172787+0000","last_became_active":"2024-04-23T01:45:16.845783+0000","last_became_peered":"2024-04-23T01:45:16.845783+0000","last_unstale":"2024-04-23T01:46:11.172787+0000","last_undegraded":"2024-04-23T01:46:11.172787+0000","last_fullsized":"2024-04-23T01:46:11.172787+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3473408,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":424,"num_write_kb":3392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":34,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.228810+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:46:11.228810+0000","last_peered":"2024-04-23T01:46:11.228810+0000","last_clean":"2024-04-23T01:46:11.228810+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:46:11.228810+0000","last_undegraded":"2024-04-23T01:46:11.228810+0000","last_fullsized":"2024-04-23T01:46:11.228810+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"22'655","reported_seq":681,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.457211+0000","last_change":"2024-04-23T01:45:16.845162+0000","last_active":"2024-04-23T01:46:11.457211+0000","last_peered":"2024-04-23T01:46:11.457211+0000","last_clean":"2024-04-23T01:46:11.457211+0000","last_became_active":"2024-04-23T01:45:16.829836+0000","last_became_peered":"2024-04-23T01:45:16.829836+0000","last_unstale":"2024-04-23T01:46:11.457211+0000","last_undegraded":"2024-04-23T01:46:11.457211+0000","last_fullsized":"2024-04-23T01:46:11.457211+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":31,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:45:35.776533+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:45:35.776533+0000","last_peered":"2024-04-23T01:45:35.776533+0000","last_clean":"2024-04-23T01:45:35.776533+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:45:35.776533+0000","last_undegraded":"2024-04-23T01:45:35.776533+0000","last_fullsized":"2024-04-23T01:45:35.776533+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"21'543","reported_seq":568,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-23T01:46:11.017341+0000","last_change":"2024-04-23T01:45:16.885391+0000","last_active":"2024-04-23T01:46:11.017341+0000","last_peered":"2024-04-23T01:46:11.017341+0000","last_clean":"2024-04-23T01:46:11.017341+0000","last_became_active":"2024-04-23T01:45:16.880317+0000","last_became_peered":"2024-04-23T01:45:16.880317+0000","last_unstale":"2024-04-23T01:46:11.017341+0000","last_undegraded":"2024-04-23T01:46:11.017341+0000","last_fullsized":"2024-04-23T01:46:11.017341+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":544,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":30,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-23T01:45:40.859361+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:45:40.859361+0000","last_peered":"2024-04-23T01:45:40.859361+0000","last_clean":"2024-04-23T01:45:40.859361+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:45:40.859361+0000","last_undegraded":"2024-04-23T01:45:40.859361+0000","last_fullsized":"2024-04-23T01:45:40.859361+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":68698112,"num_objects":1050,"num_object_clones":0,"num_object_copies":3150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8386,"num_write_kb":67088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":103030784,"data_stored":206061568,"data_compressed":10636812,"data_compressed_allocated":103030784,"data_compressed_original":206061568,"omap_allocated":0,"internal_metadata":0},"log_size":8375,"ondisk_log_size":8375,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705728,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":68116,"kb_used_data":28812,"kb_used_omap":0,"kb_used_meta":39296,"kb_avail":94303724,"statfs":{"total":96636764160,"available":96567013376,"internally_reserved":0,"allocated":29503488,"data_stored":59176409,"data_compressed":3051370,"data_compressed_allocated":29306880,"data_compressed_original":59105280,"omap_allocated":0,"internal_metadata":40239104},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705728,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58244,"kb_used_data":23100,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94313596,"statfs":{"total":96636764160,"available":96577122304,"internally_reserved":0,"allocated":23654400,"data_stored":47478233,"data_compressed":2447514,"data_compressed_allocated":23457792,"data_compressed_original":47407104,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705728,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60532,"kb_used_data":25388,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94311308,"statfs":{"total":96636764160,"available":96574779392,"internally_reserved":0,"allocated":25997312,"data_stored":51696569,"data_compressed":2665674,"data_compressed_allocated":25821184,"data_compressed_original":51642368,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705728,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":53436,"kb_used_data":22452,"kb_used_omap":0,"kb_used_meta":30976,"kb_avail":94318404,"statfs":{"total":96636764160,"available":96582045696,"internally_reserved":0,"allocated":22990848,"data_stored":46151129,"data_compressed":2378954,"data_compressed_allocated":22794240,"data_compressed_original":46080000,"omap_allocated":0,"internal_metadata":31719424},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":23195648,"data_stored":46391296,"data_compressed":2394685,"data_compressed_allocated":23195648,"data_compressed_original":46391296,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26198016,"data_stored":52396032,"data_compressed":2704590,"data_compressed_allocated":26198016,"data_compressed_original":52396032,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":23883776,"data_stored":47767552,"data_compressed":2465783,"data_compressed_allocated":23883776,"data_compressed_original":47767552,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":29753344,"data_stored":59506688,"data_compressed":3071754,"data_compressed_allocated":29753344,"data_compressed_original":59506688,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:46:15.140 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-23T01:46:15.140 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:46:15.141 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:46:15.141 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:46:15.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:15.265+0000 7f7ea739d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:15.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:15.279+0000 7f7ea739d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:15.285 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:46:15.310 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:46:15.310 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:46:15.310 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:46:15.310 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:46:15.311 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:46:15.311 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:46:15.311 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:46:15.311 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:46:15.311 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:46:15.311 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:46:15.311 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:46:15.311 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:46:15.314 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:46:15.315 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:46:15.315 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:46:15.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:15.425+0000 7faa7f664640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:15.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:15.435+0000 7faa7f664640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:15.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:15.499+0000 7f338d86d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:15.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:15.510+0000 7f338d86d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:15.538 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:46:15.538 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:46:15.538 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:46:15.538 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:46:15.538 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:46:15.538 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:46:15.538 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:46:15.538 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:46:15.538 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:46:15.538 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:46:15.539 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:46:15.539 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:46:15.540 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:46:15.541 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:46:15.541 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:46:15.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:15.662+0000 7ff01215d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:15.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:15.671+0000 7ff01215d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:15.692 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:46:15.692 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:46:15.692 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:46:15.692 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:46:15.692 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:46:15.692 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:46:15.692 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:46:15.692 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:46:15.692 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:46:15.692 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:46:15.692 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:46:15.692 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:46:15.696 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:46:15.696 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:46:15.697 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:46:15.794 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 58 16 8976 8960 1.20679 1.08594 0.173639 0.103454 2024-04-23T01:46:15.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:15.813+0000 7f8233c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:15.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:15.825+0000 7f8233c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:15.901 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:46:15.901 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:46:15.901 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:46:15.902 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:46:15.902 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:46:15.902 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:46:15.902 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:46:15.902 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:46:15.902 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:46:15.902 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:46:15.902 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:46:15.902 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:46:15.904 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:46:15.904 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:46:16.794 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 59 16 9116 9100 1.20487 1.09375 0.0992639 0.103632 2024-04-23T01:46:17.595 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:46:17.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:17.730+0000 7f075002d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:17.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:17.737+0000 7f075002d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:17.794 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:46:17.796649+0000 min lat: 0.0153353 max lat: 0.377697 avg lat: 0.103836 2024-04-23T01:46:17.794 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:46:17.794 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 60 16 9246 9230 1.20172 1.01562 0.0339667 0.103836 2024-04-23T01:46:18.794 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 61 16 9401 9385 1.20187 1.21094 0.0396215 0.103901 2024-04-23T01:46:19.795 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 62 16 9551 9535 1.20138 1.17188 0.0868778 0.103931 2024-04-23T01:46:19.934 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:46:20.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:20.053+0000 7fdd3ed9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:20.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:20.059+0000 7fdd3ed9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:20.795 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 63 16 9674 9658 1.19756 0.960938 0.171465 0.104275 2024-04-23T01:46:20.906 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:46:20.906 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:46:20.906 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:46:20.906 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:46:21.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:21.042+0000 7efd9255d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:21.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:21.055+0000 7efd9255d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:21.074 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:46:21.074 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:46:21.074 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:46:21.074 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:46:21.074 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:46:21.074 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:46:21.074 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:46:21.074 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:46:21.074 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:46:21.074 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:46:21.074 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:46:21.074 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:46:21.077 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:46:21.078 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:46:21.078 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:46:21.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:21.213+0000 7f210735d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:21.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:21.225+0000 7f210735d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:21.242 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:46:21.242 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:46:21.242 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:46:21.242 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:46:21.242 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:46:21.242 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:46:21.242 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:46:21.242 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:46:21.242 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:46:21.242 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:46:21.242 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:46:21.242 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:46:21.246 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:46:21.246 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:46:21.246 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:46:21.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:21.363+0000 7f01890ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:21.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:21.373+0000 7f01890ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:21.405 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:46:21.405 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:46:21.406 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:46:21.406 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:46:21.406 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:46:21.406 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:46:21.406 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:46:21.406 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:46:21.406 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:46:21.406 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:46:21.406 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:46:21.406 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:46:21.409 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:46:21.409 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:46:21.410 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:46:21.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:21.526+0000 7f9c64c2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:21.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:21.539+0000 7f9c64c2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:21.570 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:46:21.570 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:46:21.570 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:46:21.570 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:46:21.570 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:46:21.570 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:46:21.570 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:46:21.571 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:46:21.571 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:46:21.571 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:46:21.571 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:46:21.571 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:46:21.573 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:46:21.573 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:46:21.795 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 64 16 9810 9794 1.19545 1.0625 0.173208 0.104455 2024-04-23T01:46:22.209 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:46:22.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:22.341+0000 7f79b3e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:22.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:22.347+0000 7f79b3e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:22.795 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 65 16 9944 9928 1.19317 1.04688 0.115079 0.104653 2024-04-23T01:46:23.796 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 66 16 10092 10076 1.1926 1.15625 0.0563886 0.104682 2024-04-23T01:46:24.558 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:46:24.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:24.719+0000 7f816819d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:24.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:24.726+0000 7f816819d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:24.795 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 67 16 10248 10232 1.19299 1.21875 0.0979737 0.104662 2024-04-23T01:46:25.795 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 68 16 10405 10389 1.19348 1.22656 0.126115 0.104651 2024-04-23T01:46:26.574 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:46:26.574 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:46:26.574 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:46:26.574 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:46:26.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:26.699+0000 7f182959d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:26.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:26.709+0000 7f182959d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:26.724 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:46:26.724 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:46:26.724 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:46:26.724 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:46:26.724 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:46:26.724 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:46:26.724 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:46:26.725 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:46:26.725 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:46:26.725 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:46:26.725 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:46:26.725 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:46:26.728 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:46:26.728 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:46:26.728 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:46:26.795 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 69 16 10543 10527 1.19181 1.07812 0.0381357 0.10478 2024-04-23T01:46:26.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:26.881+0000 7fb0fe06d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:26.891 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:46:26.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:26.895+0000 7fb0fe06d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:26.922 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:46:26.922 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:46:26.923 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:46:26.923 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:46:26.923 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:46:26.923 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:46:26.923 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:46:26.923 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:46:26.923 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:46:26.923 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:46:26.923 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:46:26.923 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:46:26.926 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:46:26.926 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:46:26.926 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:46:27.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:27.028+0000 7f27da59d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:27.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:27.038+0000 7f27da59d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:27.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:27.107+0000 7fe12731d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:27.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:27.122+0000 7fe12731d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:27.144 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:46:27.145 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:46:27.145 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:46:27.145 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:46:27.145 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:46:27.145 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:46:27.145 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:46:27.145 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:46:27.145 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:46:27.145 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:46:27.145 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:46:27.145 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:46:27.148 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:46:27.148 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:46:27.148 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:46:27.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:27.275+0000 7f480a66d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:27.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:27.285+0000 7f480a66d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:27.306 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:46:27.306 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:46:27.306 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:46:27.306 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:46:27.306 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:46:27.306 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:46:27.306 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:46:27.306 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:46:27.307 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:46:27.307 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:46:27.307 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:46:27.307 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:46:27.309 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:46:27.310 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:46:27.795 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 70 16 10681 10665 1.19019 1.07812 0.132192 0.104923 2024-04-23T01:46:28.795 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 71 16 10824 10808 1.18916 1.11719 0.145847 0.10499 2024-04-23T01:46:29.232 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:46:29.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:29.371+0000 7fcaf286d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:29.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:29.380+0000 7fcaf286d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:29.795 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 72 16 10958 10942 1.18718 1.04688 0.0564176 0.105181 2024-04-23T01:46:30.795 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 73 16 11086 11070 1.18461 1 0.0428384 0.105411 2024-04-23T01:46:31.588 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:46:31.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:31.723+0000 7f8268a64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:31.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:31.731+0000 7f8268a64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:31.796 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 74 16 11229 11213 1.1837 1.11719 0.0351093 0.1055 2024-04-23T01:46:32.310 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:46:32.310 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:46:32.311 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:46:32.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:32.434+0000 7f49886ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:32.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:32.444+0000 7f49886ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:32.672 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:46:32.673 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:46:32.689 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-04-23T01:46:31.564054+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":86630944,"num_objects":1317,"num_object_clones":0,"num_object_copies":3951,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":10640,"num_write_kb":85918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10584,"ondisk_log_size":10584,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":274280,"kb_used_data":125384,"kb_used_omap":0,"kb_used_meta":148864,"kb_avail":377213080,"statfs":{"total":386547056640,"available":386266193920,"internally_reserved":0,"allocated":128393216,"data_stored":256786452,"data_compressed":13240560,"data_compressed_allocated":127496192,"data_compressed_original":256466944,"omap_allocated":0,"internal_metadata":152436736},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11730944,"num_objects":179,"num_object_clones":0,"num_object_copies":537,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1432,"num_write_kb":11456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002255"},"pg_stats":[{"pgid":"3.f","version":"25'519","reported_seq":552,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:26.287177+0000","last_change":"2024-04-23T01:45:16.856566+0000","last_active":"2024-04-23T01:46:26.287177+0000","last_peered":"2024-04-23T01:46:26.287177+0000","last_clean":"2024-04-23T01:46:26.287177+0000","last_became_active":"2024-04-23T01:45:16.848194+0000","last_became_peered":"2024-04-23T01:45:16.848194+0000","last_unstale":"2024-04-23T01:46:26.287177+0000","last_undegraded":"2024-04-23T01:46:26.287177+0000","last_fullsized":"2024-04-23T01:46:26.287177+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259840,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":520,"num_write_kb":4160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"25'687","reported_seq":724,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:24.882345+0000","last_change":"2024-04-23T01:45:16.906007+0000","last_active":"2024-04-23T01:46:24.882345+0000","last_peered":"2024-04-23T01:46:24.882345+0000","last_clean":"2024-04-23T01:46:24.882345+0000","last_became_active":"2024-04-23T01:45:16.901215+0000","last_became_peered":"2024-04-23T01:45:16.901215+0000","last_unstale":"2024-04-23T01:46:24.882345+0000","last_undegraded":"2024-04-23T01:46:24.882345+0000","last_fullsized":"2024-04-23T01:46:24.882345+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5636096,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":688,"num_write_kb":5504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"25'623","reported_seq":656,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:26.373648+0000","last_change":"2024-04-23T01:45:16.886628+0000","last_active":"2024-04-23T01:46:26.373648+0000","last_peered":"2024-04-23T01:46:26.373648+0000","last_clean":"2024-04-23T01:46:26.373648+0000","last_became_active":"2024-04-23T01:45:16.880681+0000","last_became_peered":"2024-04-23T01:45:16.880681+0000","last_unstale":"2024-04-23T01:46:26.373648+0000","last_undegraded":"2024-04-23T01:46:26.373648+0000","last_fullsized":"2024-04-23T01:46:26.373648+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5111808,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":624,"num_write_kb":4992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"25'693","reported_seq":726,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:26.541516+0000","last_change":"2024-04-23T01:45:16.836330+0000","last_active":"2024-04-23T01:46:26.541516+0000","last_peered":"2024-04-23T01:46:26.541516+0000","last_clean":"2024-04-23T01:46:26.541516+0000","last_became_active":"2024-04-23T01:45:16.821451+0000","last_became_peered":"2024-04-23T01:45:16.821451+0000","last_unstale":"2024-04-23T01:46:26.541516+0000","last_undegraded":"2024-04-23T01:46:26.541516+0000","last_fullsized":"2024-04-23T01:46:26.541516+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":693,"log_dups_size":0,"ondisk_log_size":693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5685248,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":694,"num_write_kb":5552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"25'591","reported_seq":624,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:26.414375+0000","last_change":"2024-04-23T01:45:16.860867+0000","last_active":"2024-04-23T01:46:26.414375+0000","last_peered":"2024-04-23T01:46:26.414375+0000","last_clean":"2024-04-23T01:46:26.414375+0000","last_became_active":"2024-04-23T01:45:16.852271+0000","last_became_peered":"2024-04-23T01:45:16.852271+0000","last_unstale":"2024-04-23T01:46:26.414375+0000","last_undegraded":"2024-04-23T01:46:26.414375+0000","last_fullsized":"2024-04-23T01:46:26.414375+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":592,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"25'647","reported_seq":680,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:26.451202+0000","last_change":"2024-04-23T01:45:16.805452+0000","last_active":"2024-04-23T01:46:26.451202+0000","last_peered":"2024-04-23T01:46:26.451202+0000","last_clean":"2024-04-23T01:46:26.451202+0000","last_became_active":"2024-04-23T01:45:16.797805+0000","last_became_peered":"2024-04-23T01:45:16.797805+0000","last_unstale":"2024-04-23T01:46:26.451202+0000","last_undegraded":"2024-04-23T01:46:26.451202+0000","last_fullsized":"2024-04-23T01:46:26.451202+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308416,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":648,"num_write_kb":5184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"25'719","reported_seq":752,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:26.551448+0000","last_change":"2024-04-23T01:45:16.893894+0000","last_active":"2024-04-23T01:46:26.551448+0000","last_peered":"2024-04-23T01:46:26.551448+0000","last_clean":"2024-04-23T01:46:26.551448+0000","last_became_active":"2024-04-23T01:45:16.888673+0000","last_became_peered":"2024-04-23T01:45:16.888673+0000","last_unstale":"2024-04-23T01:46:26.551448+0000","last_undegraded":"2024-04-23T01:46:26.551448+0000","last_fullsized":"2024-04-23T01:46:26.551448+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":720,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"25'696","reported_seq":729,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:25.774570+0000","last_change":"2024-04-23T01:45:16.889162+0000","last_active":"2024-04-23T01:46:25.774570+0000","last_peered":"2024-04-23T01:46:25.774570+0000","last_clean":"2024-04-23T01:46:25.774570+0000","last_became_active":"2024-04-23T01:45:16.880761+0000","last_became_peered":"2024-04-23T01:45:16.880761+0000","last_unstale":"2024-04-23T01:46:25.774570+0000","last_undegraded":"2024-04-23T01:46:25.774570+0000","last_fullsized":"2024-04-23T01:46:25.774570+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5709824,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":697,"num_write_kb":5576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":174,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.777638+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:46:14.777638+0000","last_peered":"2024-04-23T01:46:14.777638+0000","last_clean":"2024-04-23T01:46:14.777638+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:46:14.777638+0000","last_undegraded":"2024-04-23T01:46:14.777638+0000","last_fullsized":"2024-04-23T01:46:14.777638+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.815521+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:46:14.815521+0000","last_peered":"2024-04-23T01:46:14.815521+0000","last_clean":"2024-04-23T01:46:14.815521+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:46:14.815521+0000","last_undegraded":"2024-04-23T01:46:14.815521+0000","last_fullsized":"2024-04-23T01:46:14.815521+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"25'631","reported_seq":664,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:25.978220+0000","last_change":"2024-04-23T01:45:16.891787+0000","last_active":"2024-04-23T01:46:25.978220+0000","last_peered":"2024-04-23T01:46:25.978220+0000","last_clean":"2024-04-23T01:46:25.978220+0000","last_became_active":"2024-04-23T01:45:16.885404+0000","last_became_peered":"2024-04-23T01:45:16.885404+0000","last_unstale":"2024-04-23T01:46:25.978220+0000","last_undegraded":"2024-04-23T01:46:25.978220+0000","last_fullsized":"2024-04-23T01:46:25.978220+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":5056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.256426+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:46:14.256426+0000","last_peered":"2024-04-23T01:46:14.256426+0000","last_clean":"2024-04-23T01:46:14.256426+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:46:14.256426+0000","last_undegraded":"2024-04-23T01:46:14.256426+0000","last_fullsized":"2024-04-23T01:46:14.256426+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"25'655","reported_seq":688,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:26.188974+0000","last_change":"2024-04-23T01:45:16.919259+0000","last_active":"2024-04-23T01:46:26.188974+0000","last_peered":"2024-04-23T01:46:26.188974+0000","last_clean":"2024-04-23T01:46:26.188974+0000","last_became_active":"2024-04-23T01:45:16.912983+0000","last_became_peered":"2024-04-23T01:45:16.912983+0000","last_unstale":"2024-04-23T01:46:26.188974+0000","last_undegraded":"2024-04-23T01:46:26.188974+0000","last_fullsized":"2024-04-23T01:46:26.188974+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.788785+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:46:14.788785+0000","last_peered":"2024-04-23T01:46:14.788785+0000","last_clean":"2024-04-23T01:46:14.788785+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:46:14.788785+0000","last_undegraded":"2024-04-23T01:46:14.788785+0000","last_fullsized":"2024-04-23T01:46:14.788785+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"25'604","reported_seq":637,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:26.546350+0000","last_change":"2024-04-23T01:45:16.897089+0000","last_active":"2024-04-23T01:46:26.546350+0000","last_peered":"2024-04-23T01:46:26.546350+0000","last_clean":"2024-04-23T01:46:26.546350+0000","last_became_active":"2024-04-23T01:45:16.891776+0000","last_became_peered":"2024-04-23T01:45:16.891776+0000","last_unstale":"2024-04-23T01:46:26.546350+0000","last_undegraded":"2024-04-23T01:46:26.546350+0000","last_fullsized":"2024-04-23T01:46:26.546350+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4956160,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":605,"num_write_kb":4840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.263415+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:46:14.263415+0000","last_peered":"2024-04-23T01:46:14.263415+0000","last_clean":"2024-04-23T01:46:14.263415+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:46:14.263415+0000","last_undegraded":"2024-04-23T01:46:14.263415+0000","last_fullsized":"2024-04-23T01:46:14.263415+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"25'759","reported_seq":796,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:26.204203+0000","last_change":"2024-04-23T01:45:16.823909+0000","last_active":"2024-04-23T01:46:26.204203+0000","last_peered":"2024-04-23T01:46:26.204203+0000","last_clean":"2024-04-23T01:46:26.204203+0000","last_became_active":"2024-04-23T01:45:16.815467+0000","last_became_peered":"2024-04-23T01:45:16.815467+0000","last_unstale":"2024-04-23T01:46:26.204203+0000","last_undegraded":"2024-04-23T01:46:26.204203+0000","last_fullsized":"2024-04-23T01:46:26.204203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6225920,"num_objects":95,"num_object_clones":0,"num_object_copies":285,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":760,"num_write_kb":6080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"25'687","reported_seq":720,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:26.164918+0000","last_change":"2024-04-23T01:45:16.859980+0000","last_active":"2024-04-23T01:46:26.164918+0000","last_peered":"2024-04-23T01:46:26.164918+0000","last_clean":"2024-04-23T01:46:26.164918+0000","last_became_active":"2024-04-23T01:45:16.855622+0000","last_became_peered":"2024-04-23T01:45:16.855622+0000","last_unstale":"2024-04-23T01:46:26.164918+0000","last_undegraded":"2024-04-23T01:46:26.164918+0000","last_fullsized":"2024-04-23T01:46:26.164918+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5636096,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":688,"num_write_kb":5504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.805143+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:46:14.805143+0000","last_peered":"2024-04-23T01:46:14.805143+0000","last_clean":"2024-04-23T01:46:14.805143+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:46:14.805143+0000","last_undegraded":"2024-04-23T01:46:14.805143+0000","last_fullsized":"2024-04-23T01:46:14.805143+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"25'487","reported_seq":524,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:25.626255+0000","last_change":"2024-04-23T01:45:16.854641+0000","last_active":"2024-04-23T01:46:25.626255+0000","last_peered":"2024-04-23T01:46:25.626255+0000","last_clean":"2024-04-23T01:46:25.626255+0000","last_became_active":"2024-04-23T01:45:16.845783+0000","last_became_peered":"2024-04-23T01:45:16.845783+0000","last_unstale":"2024-04-23T01:46:25.626255+0000","last_undegraded":"2024-04-23T01:46:25.626255+0000","last_fullsized":"2024-04-23T01:46:25.626255+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3997696,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":488,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.802955+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:46:14.802955+0000","last_peered":"2024-04-23T01:46:14.802955+0000","last_clean":"2024-04-23T01:46:14.802955+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:46:14.802955+0000","last_undegraded":"2024-04-23T01:46:14.802955+0000","last_fullsized":"2024-04-23T01:46:14.802955+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"25'786","reported_seq":819,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:26.546485+0000","last_change":"2024-04-23T01:45:16.845162+0000","last_active":"2024-04-23T01:46:26.546485+0000","last_peered":"2024-04-23T01:46:26.546485+0000","last_clean":"2024-04-23T01:46:26.546485+0000","last_became_active":"2024-04-23T01:45:16.829836+0000","last_became_peered":"2024-04-23T01:45:16.829836+0000","last_unstale":"2024-04-23T01:46:26.546485+0000","last_undegraded":"2024-04-23T01:46:26.546485+0000","last_fullsized":"2024-04-23T01:46:26.546485+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6447104,"num_objects":99,"num_object_clones":0,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":787,"num_write_kb":6296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":43,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.793283+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:46:14.793283+0000","last_peered":"2024-04-23T01:46:14.793283+0000","last_clean":"2024-04-23T01:46:14.793283+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:46:14.793283+0000","last_undegraded":"2024-04-23T01:46:14.793283+0000","last_fullsized":"2024-04-23T01:46:14.793283+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"25'703","reported_seq":740,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:25.394313+0000","last_change":"2024-04-23T01:45:16.885391+0000","last_active":"2024-04-23T01:46:25.394313+0000","last_peered":"2024-04-23T01:46:25.394313+0000","last_clean":"2024-04-23T01:46:25.394313+0000","last_became_active":"2024-04-23T01:45:16.880317+0000","last_became_peered":"2024-04-23T01:45:16.880317+0000","last_unstale":"2024-04-23T01:46:25.394313+0000","last_undegraded":"2024-04-23T01:46:25.394313+0000","last_fullsized":"2024-04-23T01:46:25.394313+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767168,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":704,"num_write_kb":5632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.803589+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:46:14.803589+0000","last_peered":"2024-04-23T01:46:14.803589+0000","last_clean":"2024-04-23T01:46:14.803589+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:46:14.803589+0000","last_undegraded":"2024-04-23T01:46:14.803589+0000","last_fullsized":"2024-04-23T01:46:14.803589+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":86040576,"num_objects":1315,"num_object_clones":0,"num_object_copies":3945,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10503,"num_write_kb":84024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":129048576,"data_stored":258097152,"data_compressed":13323708,"data_compressed_allocated":129048576,"data_compressed_original":258097152,"omap_allocated":0,"internal_metadata":0},"log_size":10487,"ondisk_log_size":10487,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705743,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":67308,"kb_used_data":36324,"kb_used_omap":0,"kb_used_meta":30976,"kb_avail":94304532,"statfs":{"total":96636764160,"available":96567840768,"internally_reserved":0,"allocated":37195776,"data_stored":74508429,"data_compressed":3842380,"data_compressed_allocated":36966400,"data_compressed_original":74424320,"omap_allocated":0,"internal_metadata":31719424},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705743,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":68372,"kb_used_data":29068,"kb_used_omap":0,"kb_used_meta":39296,"kb_avail":94303468,"statfs":{"total":96636764160,"available":96566751232,"internally_reserved":0,"allocated":29765632,"data_stored":59648141,"data_compressed":3075246,"data_compressed_allocated":29536256,"data_compressed_original":59564032,"omap_allocated":0,"internal_metadata":40239104},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705743,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":70976,"kb_used_data":31672,"kb_used_omap":0,"kb_used_meta":39296,"kb_avail":94300864,"statfs":{"total":96636764160,"available":96564084736,"internally_reserved":0,"allocated":32432128,"data_stored":64513645,"data_compressed":3326823,"data_compressed_allocated":32223232,"data_compressed_original":64446464,"omap_allocated":0,"internal_metadata":40239104},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705743,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":67624,"kb_used_data":28320,"kb_used_omap":0,"kb_used_meta":39296,"kb_avail":94304216,"statfs":{"total":96636764160,"available":96567517184,"internally_reserved":0,"allocated":28999680,"data_stored":58116237,"data_compressed":2996111,"data_compressed_allocated":28770304,"data_compressed_original":58032128,"omap_allocated":0,"internal_metadata":40239104},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":29167616,"data_stored":58335232,"data_compressed":3011419,"data_compressed_allocated":29167616,"data_compressed_original":58335232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32641024,"data_stored":65282048,"data_compressed":3369969,"data_compressed_allocated":32641024,"data_compressed_original":65282048,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":29827072,"data_stored":59654144,"data_compressed":3079556,"data_compressed_allocated":29827072,"data_compressed_original":59654144,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":37412864,"data_stored":74825728,"data_compressed":3862764,"data_compressed_allocated":37412864,"data_compressed_original":74825728,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:46:32.796 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 75 16 11361 11345 1.18167 1.03125 0.118771 0.105681 2024-04-23T01:46:33.796 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 76 16 11490 11474 1.17938 1.00781 0.0937576 0.105865 2024-04-23T01:46:34.018 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:46:34.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:34.151+0000 7fabb0d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:34.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:34.157+0000 7fabb0d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:34.796 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 77 16 11626 11610 1.17786 1.0625 0.147809 0.106046 2024-04-23T01:46:35.796 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 78 16 11760 11744 1.17618 1.04688 0.0789761 0.106181 2024-04-23T01:46:36.460 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:46:36.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:36.585+0000 7f1f57e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:36.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:36.592+0000 7f1f57e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:36.796 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 79 16 11890 11874 1.17415 1.01562 0.131175 0.106332 2024-04-23T01:46:37.691 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:46:37.691 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:46:37.692 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:46:37.796 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:46:37.798441+0000 min lat: 0.0153353 max lat: 0.377697 avg lat: 0.106532 2024-04-23T01:46:37.796 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:46:37.796 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 80 16 12021 12005 1.17226 1.02344 0.107485 0.106532 2024-04-23T01:46:37.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:37.816+0000 7f0f04e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:37.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:37.827+0000 7f0f04e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:38.187 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:46:38.187 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:46:38.208 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-04-23T01:46:37.565075+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":97673760,"num_objects":1485,"num_object_clones":0,"num_object_copies":4455,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":11988,"num_write_kb":96702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11932,"ondisk_log_size":11932,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":257324,"kb_used_data":141708,"kb_used_omap":0,"kb_used_meta":115584,"kb_avail":377230036,"statfs":{"total":386547056640,"available":386283556864,"internally_reserved":0,"allocated":145108992,"data_stored":290218004,"data_compressed":14966820,"data_compressed_allocated":144211968,"data_compressed_original":289898496,"omap_allocated":0,"internal_metadata":118358016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17072128,"num_objects":260,"num_object_clones":0,"num_object_copies":780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2084,"num_write_kb":16672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002098"},"pg_stats":[{"pgid":"3.f","version":"25'628","reported_seq":661,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:36.543147+0000","last_change":"2024-04-23T01:45:16.856566+0000","last_active":"2024-04-23T01:46:36.543147+0000","last_peered":"2024-04-23T01:46:36.543147+0000","last_clean":"2024-04-23T01:46:36.543147+0000","last_became_active":"2024-04-23T01:45:16.848194+0000","last_became_peered":"2024-04-23T01:45:16.848194+0000","last_unstale":"2024-04-23T01:46:36.543147+0000","last_undegraded":"2024-04-23T01:46:36.543147+0000","last_fullsized":"2024-04-23T01:46:36.543147+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5152768,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":629,"num_write_kb":5032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"25'743","reported_seq":780,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:36.450795+0000","last_change":"2024-04-23T01:45:16.906007+0000","last_active":"2024-04-23T01:46:36.450795+0000","last_peered":"2024-04-23T01:46:36.450795+0000","last_clean":"2024-04-23T01:46:36.450795+0000","last_became_active":"2024-04-23T01:45:16.901215+0000","last_became_peered":"2024-04-23T01:45:16.901215+0000","last_unstale":"2024-04-23T01:46:36.450795+0000","last_undegraded":"2024-04-23T01:46:36.450795+0000","last_fullsized":"2024-04-23T01:46:36.450795+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6094848,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":744,"num_write_kb":5952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"25'663","reported_seq":696,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:36.026764+0000","last_change":"2024-04-23T01:45:16.886628+0000","last_active":"2024-04-23T01:46:36.026764+0000","last_peered":"2024-04-23T01:46:36.026764+0000","last_clean":"2024-04-23T01:46:36.026764+0000","last_became_active":"2024-04-23T01:45:16.880681+0000","last_became_peered":"2024-04-23T01:45:16.880681+0000","last_unstale":"2024-04-23T01:46:36.026764+0000","last_undegraded":"2024-04-23T01:46:36.026764+0000","last_fullsized":"2024-04-23T01:46:36.026764+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5439488,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":664,"num_write_kb":5312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"25'776","reported_seq":809,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:33.467739+0000","last_change":"2024-04-23T01:45:16.836330+0000","last_active":"2024-04-23T01:46:33.467739+0000","last_peered":"2024-04-23T01:46:33.467739+0000","last_clean":"2024-04-23T01:46:33.467739+0000","last_became_active":"2024-04-23T01:45:16.821451+0000","last_became_peered":"2024-04-23T01:45:16.821451+0000","last_unstale":"2024-04-23T01:46:33.467739+0000","last_undegraded":"2024-04-23T01:46:33.467739+0000","last_fullsized":"2024-04-23T01:46:33.467739+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":776,"log_dups_size":0,"ondisk_log_size":776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6365184,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":777,"num_write_kb":6216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"25'671","reported_seq":704,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:34.975205+0000","last_change":"2024-04-23T01:45:16.860867+0000","last_active":"2024-04-23T01:46:34.975205+0000","last_peered":"2024-04-23T01:46:34.975205+0000","last_clean":"2024-04-23T01:46:34.975205+0000","last_became_active":"2024-04-23T01:45:16.852271+0000","last_became_peered":"2024-04-23T01:45:16.852271+0000","last_unstale":"2024-04-23T01:46:34.975205+0000","last_undegraded":"2024-04-23T01:46:34.975205+0000","last_fullsized":"2024-04-23T01:46:34.975205+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505024,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":672,"num_write_kb":5376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"25'743","reported_seq":776,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:35.073969+0000","last_change":"2024-04-23T01:45:16.805452+0000","last_active":"2024-04-23T01:46:35.073969+0000","last_peered":"2024-04-23T01:46:35.073969+0000","last_clean":"2024-04-23T01:46:35.073969+0000","last_became_active":"2024-04-23T01:45:16.797805+0000","last_became_peered":"2024-04-23T01:45:16.797805+0000","last_unstale":"2024-04-23T01:46:35.073969+0000","last_undegraded":"2024-04-23T01:46:35.073969+0000","last_fullsized":"2024-04-23T01:46:35.073969+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6094848,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":744,"num_write_kb":5952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"25'789","reported_seq":822,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:36.545116+0000","last_change":"2024-04-23T01:45:16.893894+0000","last_active":"2024-04-23T01:46:36.545116+0000","last_peered":"2024-04-23T01:46:36.545116+0000","last_clean":"2024-04-23T01:46:36.545116+0000","last_became_active":"2024-04-23T01:45:16.888673+0000","last_became_peered":"2024-04-23T01:45:16.888673+0000","last_unstale":"2024-04-23T01:46:36.545116+0000","last_undegraded":"2024-04-23T01:46:36.545116+0000","last_fullsized":"2024-04-23T01:46:36.545116+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":789,"log_dups_size":0,"ondisk_log_size":789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6471680,"num_objects":99,"num_object_clones":0,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":790,"num_write_kb":6320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"25'783","reported_seq":816,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:36.040426+0000","last_change":"2024-04-23T01:45:16.889162+0000","last_active":"2024-04-23T01:46:36.040426+0000","last_peered":"2024-04-23T01:46:36.040426+0000","last_clean":"2024-04-23T01:46:36.040426+0000","last_became_active":"2024-04-23T01:45:16.880761+0000","last_became_peered":"2024-04-23T01:45:16.880761+0000","last_unstale":"2024-04-23T01:46:36.040426+0000","last_undegraded":"2024-04-23T01:46:36.040426+0000","last_fullsized":"2024-04-23T01:46:36.040426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6422528,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":174,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.777638+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:46:14.777638+0000","last_peered":"2024-04-23T01:46:14.777638+0000","last_clean":"2024-04-23T01:46:14.777638+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:46:14.777638+0000","last_undegraded":"2024-04-23T01:46:14.777638+0000","last_fullsized":"2024-04-23T01:46:14.777638+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.815521+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:46:14.815521+0000","last_peered":"2024-04-23T01:46:14.815521+0000","last_clean":"2024-04-23T01:46:14.815521+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:46:14.815521+0000","last_undegraded":"2024-04-23T01:46:14.815521+0000","last_fullsized":"2024-04-23T01:46:14.815521+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"25'727","reported_seq":760,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:36.380083+0000","last_change":"2024-04-23T01:45:16.891787+0000","last_active":"2024-04-23T01:46:36.380083+0000","last_peered":"2024-04-23T01:46:36.380083+0000","last_clean":"2024-04-23T01:46:36.380083+0000","last_became_active":"2024-04-23T01:45:16.885404+0000","last_became_peered":"2024-04-23T01:45:16.885404+0000","last_unstale":"2024-04-23T01:46:36.380083+0000","last_undegraded":"2024-04-23T01:46:36.380083+0000","last_fullsized":"2024-04-23T01:46:36.380083+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":728,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.256426+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:46:14.256426+0000","last_peered":"2024-04-23T01:46:14.256426+0000","last_clean":"2024-04-23T01:46:14.256426+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:46:14.256426+0000","last_undegraded":"2024-04-23T01:46:14.256426+0000","last_fullsized":"2024-04-23T01:46:14.256426+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"25'759","reported_seq":792,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:36.444084+0000","last_change":"2024-04-23T01:45:16.919259+0000","last_active":"2024-04-23T01:46:36.444084+0000","last_peered":"2024-04-23T01:46:36.444084+0000","last_clean":"2024-04-23T01:46:36.444084+0000","last_became_active":"2024-04-23T01:45:16.912983+0000","last_became_peered":"2024-04-23T01:45:16.912983+0000","last_unstale":"2024-04-23T01:46:36.444084+0000","last_undegraded":"2024-04-23T01:46:36.444084+0000","last_fullsized":"2024-04-23T01:46:36.444084+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6225920,"num_objects":95,"num_object_clones":0,"num_object_copies":285,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":760,"num_write_kb":6080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.788785+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:46:14.788785+0000","last_peered":"2024-04-23T01:46:14.788785+0000","last_clean":"2024-04-23T01:46:14.788785+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:46:14.788785+0000","last_undegraded":"2024-04-23T01:46:14.788785+0000","last_fullsized":"2024-04-23T01:46:14.788785+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"25'647","reported_seq":680,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:35.739607+0000","last_change":"2024-04-23T01:45:16.897089+0000","last_active":"2024-04-23T01:46:35.739607+0000","last_peered":"2024-04-23T01:46:35.739607+0000","last_clean":"2024-04-23T01:46:35.739607+0000","last_became_active":"2024-04-23T01:45:16.891776+0000","last_became_peered":"2024-04-23T01:45:16.891776+0000","last_unstale":"2024-04-23T01:46:35.739607+0000","last_undegraded":"2024-04-23T01:46:35.739607+0000","last_fullsized":"2024-04-23T01:46:35.739607+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308416,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":648,"num_write_kb":5184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.263415+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:46:14.263415+0000","last_peered":"2024-04-23T01:46:14.263415+0000","last_clean":"2024-04-23T01:46:14.263415+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:46:14.263415+0000","last_undegraded":"2024-04-23T01:46:14.263415+0000","last_fullsized":"2024-04-23T01:46:14.263415+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"25'855","reported_seq":892,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:35.975089+0000","last_change":"2024-04-23T01:45:16.823909+0000","last_active":"2024-04-23T01:46:35.975089+0000","last_peered":"2024-04-23T01:46:35.975089+0000","last_clean":"2024-04-23T01:46:35.975089+0000","last_became_active":"2024-04-23T01:45:16.815467+0000","last_became_peered":"2024-04-23T01:45:16.815467+0000","last_unstale":"2024-04-23T01:46:35.975089+0000","last_undegraded":"2024-04-23T01:46:35.975089+0000","last_fullsized":"2024-04-23T01:46:35.975089+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7012352,"num_objects":107,"num_object_clones":0,"num_object_copies":321,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":856,"num_write_kb":6848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"25'775","reported_seq":808,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:35.610275+0000","last_change":"2024-04-23T01:45:16.859980+0000","last_active":"2024-04-23T01:46:35.610275+0000","last_peered":"2024-04-23T01:46:35.610275+0000","last_clean":"2024-04-23T01:46:35.610275+0000","last_became_active":"2024-04-23T01:45:16.855622+0000","last_became_peered":"2024-04-23T01:45:16.855622+0000","last_unstale":"2024-04-23T01:46:35.610275+0000","last_undegraded":"2024-04-23T01:46:35.610275+0000","last_fullsized":"2024-04-23T01:46:35.610275+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6356992,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":776,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.805143+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:46:14.805143+0000","last_peered":"2024-04-23T01:46:14.805143+0000","last_clean":"2024-04-23T01:46:14.805143+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:46:14.805143+0000","last_undegraded":"2024-04-23T01:46:14.805143+0000","last_fullsized":"2024-04-23T01:46:14.805143+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"25'606","reported_seq":643,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:36.533123+0000","last_change":"2024-04-23T01:45:16.854641+0000","last_active":"2024-04-23T01:46:36.533123+0000","last_peered":"2024-04-23T01:46:36.533123+0000","last_clean":"2024-04-23T01:46:36.533123+0000","last_became_active":"2024-04-23T01:45:16.845783+0000","last_became_peered":"2024-04-23T01:45:16.845783+0000","last_unstale":"2024-04-23T01:46:36.533123+0000","last_undegraded":"2024-04-23T01:46:36.533123+0000","last_fullsized":"2024-04-23T01:46:36.533123+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4972544,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":607,"num_write_kb":4856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.802955+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:46:14.802955+0000","last_peered":"2024-04-23T01:46:14.802955+0000","last_clean":"2024-04-23T01:46:14.802955+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:46:14.802955+0000","last_undegraded":"2024-04-23T01:46:14.802955+0000","last_fullsized":"2024-04-23T01:46:14.802955+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"25'903","reported_seq":936,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:36.100215+0000","last_change":"2024-04-23T01:45:16.845162+0000","last_active":"2024-04-23T01:46:36.100215+0000","last_peered":"2024-04-23T01:46:36.100215+0000","last_clean":"2024-04-23T01:46:36.100215+0000","last_became_active":"2024-04-23T01:45:16.829836+0000","last_became_peered":"2024-04-23T01:45:16.829836+0000","last_unstale":"2024-04-23T01:46:36.100215+0000","last_undegraded":"2024-04-23T01:46:36.100215+0000","last_fullsized":"2024-04-23T01:46:36.100215+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":903,"log_dups_size":0,"ondisk_log_size":903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7405568,"num_objects":113,"num_object_clones":0,"num_object_copies":339,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":904,"num_write_kb":7232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":43,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.793283+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:46:14.793283+0000","last_peered":"2024-04-23T01:46:14.793283+0000","last_clean":"2024-04-23T01:46:14.793283+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:46:14.793283+0000","last_undegraded":"2024-04-23T01:46:14.793283+0000","last_fullsized":"2024-04-23T01:46:14.793283+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"25'767","reported_seq":804,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:35.346011+0000","last_change":"2024-04-23T01:45:16.885391+0000","last_active":"2024-04-23T01:46:35.346011+0000","last_peered":"2024-04-23T01:46:35.346011+0000","last_clean":"2024-04-23T01:46:35.346011+0000","last_became_active":"2024-04-23T01:45:16.880317+0000","last_became_peered":"2024-04-23T01:45:16.880317+0000","last_unstale":"2024-04-23T01:46:35.346011+0000","last_undegraded":"2024-04-23T01:46:35.346011+0000","last_fullsized":"2024-04-23T01:46:35.346011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":768,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.803589+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:46:14.803589+0000","last_peered":"2024-04-23T01:46:14.803589+0000","last_clean":"2024-04-23T01:46:14.803589+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:46:14.803589+0000","last_undegraded":"2024-04-23T01:46:14.803589+0000","last_fullsized":"2024-04-23T01:46:14.803589+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":97083392,"num_objects":1483,"num_object_clones":0,"num_object_copies":4449,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11851,"num_write_kb":94808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":145616896,"data_stored":291233792,"data_compressed":15034740,"data_compressed_allocated":145616896,"data_compressed_original":291233792,"omap_allocated":0,"internal_metadata":0},"log_size":11835,"ondisk_log_size":11835,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705753,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":76212,"kb_used_data":41068,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94295628,"statfs":{"total":96636764160,"available":96558723072,"internally_reserved":0,"allocated":42053632,"data_stored":84224141,"data_compressed":4344057,"data_compressed_allocated":41824256,"data_compressed_original":84140032,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705753,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59456,"kb_used_data":32632,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94312384,"statfs":{"total":96636764160,"available":96575881216,"internally_reserved":0,"allocated":33415168,"data_stored":66947213,"data_compressed":3452139,"data_compressed_allocated":33185792,"data_compressed_original":66863104,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705753,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":62376,"kb_used_data":35552,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94309464,"statfs":{"total":96636764160,"available":96572891136,"internally_reserved":0,"allocated":36405248,"data_stored":72459885,"data_compressed":3737132,"data_compressed_allocated":36196352,"data_compressed_original":72392704,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705753,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59280,"kb_used_data":32456,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94312560,"statfs":{"total":96636764160,"available":96576061440,"internally_reserved":0,"allocated":33234944,"data_stored":66586765,"data_compressed":3433492,"data_compressed_allocated":33005568,"data_compressed_original":66502656,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":33300480,"data_stored":66600960,"data_compressed":3438225,"data_compressed_allocated":33300480,"data_compressed_original":66600960,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36614144,"data_stored":73228288,"data_compressed":3780278,"data_compressed_allocated":36614144,"data_compressed_original":73228288,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":33447936,"data_stored":66895872,"data_compressed":3453488,"data_compressed_allocated":33447936,"data_compressed_original":66895872,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":42254336,"data_stored":84508672,"data_compressed":4362749,"data_compressed_allocated":42254336,"data_compressed_original":84508672,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:46:38.796 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 81 16 12158 12142 1.171 1.07031 0.11181 0.106663 2024-04-23T01:46:38.873 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:46:39.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:39.009+0000 7f63cbf5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:39.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:39.016+0000 7f63cbf5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:39.797 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 82 16 12288 12272 1.1691 1.01562 0.0975656 0.106844 2024-04-23T01:46:40.797 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 83 16 12425 12409 1.16791 1.07031 0.0647244 0.106893 2024-04-23T01:46:41.209 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:46:41.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:41.350+0000 7fa3d611d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:41.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:41.359+0000 7fa3d611d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:41.797 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 84 16 12561 12545 1.16666 1.0625 0.0926162 0.107007 2024-04-23T01:46:42.797 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 85 16 12680 12664 1.16387 0.929688 0.0372676 0.107118 2024-04-23T01:46:43.211 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:46:43.211 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:46:43.211 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:46:43.211 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:46:43.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:43.351+0000 7fb1e439d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:43.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:43.362+0000 7fb1e439d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:43.378 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:46:43.378 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:46:43.379 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:46:43.379 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:46:43.379 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:46:43.379 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:46:43.379 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:46:43.379 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:46:43.379 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:46:43.379 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:46:43.379 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:46:43.379 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:46:43.382 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:46:43.382 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:46:43.382 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:46:43.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:43.517+0000 7faabed5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:43.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:43.528+0000 7faabed5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:43.529 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:46:43.560 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:46:43.560 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:46:43.560 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:46:43.560 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:46:43.560 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:46:43.560 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:46:43.560 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:46:43.560 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:46:43.560 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:46:43.560 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:46:43.560 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:46:43.561 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:46:43.564 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:46:43.564 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:46:43.564 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:46:43.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:43.671+0000 7f35d2c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:43.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:43.678+0000 7f35d2c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:43.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:43.754+0000 7f722aced640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:43.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:43.769+0000 7f722aced640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:43.797 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 86 16 12816 12800 1.16269 1.0625 0.116571 0.107402 2024-04-23T01:46:43.804 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:46:43.804 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:46:43.804 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:46:43.804 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:46:43.804 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:46:43.804 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:46:43.804 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:46:43.804 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:46:43.804 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:46:43.804 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:46:43.804 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:46:43.804 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:46:43.808 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:46:43.808 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:46:43.808 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:46:43.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:43.956+0000 7f4cd6e2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:43.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:43.967+0000 7f4cd6e2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:43.984 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:46:43.984 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:46:43.984 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:46:43.984 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:46:43.984 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:46:43.984 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:46:43.984 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:46:43.984 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:46:43.984 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:46:43.985 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:46:43.985 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:46:43.985 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:46:43.988 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:46:43.988 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:46:44.797 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 87 16 12953 12937 1.16162 1.07031 0.127922 0.107514 2024-04-23T01:46:45.797 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 88 16 13089 13073 1.1605 1.0625 0.0748357 0.107587 2024-04-23T01:46:45.874 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:46:46.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:46.000+0000 7f35d6ced640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:46.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:46.009+0000 7f35d6ced640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:46.797 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 89 16 13227 13211 1.15957 1.07812 0.16564 0.107696 2024-04-23T01:46:47.797 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 90 15 13361 13346 1.1584 1.05469 0.0730144 0.10782 2024-04-23T01:46:48.187 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:46:48.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:48.292+0000 7fb2c8264640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:48.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:48.298+0000 7fb2c8264640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:48.797 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Total time run: 90.1028 2024-04-23T01:46:48.798 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Total writes made: 13361 2024-04-23T01:46:48.798 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Write size: 8192 2024-04-23T01:46:48.798 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Object size: 65536 2024-04-23T01:46:48.798 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Bandwidth (MB/sec): 1.15849 2024-04-23T01:46:48.798 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Stddev Bandwidth: 0.106672 2024-04-23T01:46:48.798 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Max bandwidth (MB/sec): 1.42969 2024-04-23T01:46:48.798 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Min bandwidth (MB/sec): 0.929688 2024-04-23T01:46:48.798 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Average IOPS: 148 2024-04-23T01:46:48.798 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Stddev IOPS: 13.654 2024-04-23T01:46:48.798 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Max IOPS: 183 2024-04-23T01:46:48.798 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Min IOPS: 119 2024-04-23T01:46:48.799 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Average Latency(s): 0.10783 2024-04-23T01:46:48.799 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Stddev Latency(s): 0.0567301 2024-04-23T01:46:48.799 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Max latency(s): 0.435613 2024-04-23T01:46:48.799 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Min latency(s): 0.0153353 2024-04-23T01:46:48.820 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Cleaning up (deleting benchmark objects) 2024-04-23T01:46:48.989 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:46:48.989 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:46:48.989 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:46:49.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:49.131+0000 7fef98f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:49.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:49.142+0000 7fef98f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:49.389 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:46:49.389 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:46:49.412 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-04-23T01:46:47.566973+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":108601888,"num_objects":1652,"num_object_clones":0,"num_object_copies":4956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":13322,"num_write_kb":107374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13266,"ondisk_log_size":13266,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":290016,"kb_used_data":157760,"kb_used_omap":0,"kb_used_meta":132224,"kb_avail":377197344,"statfs":{"total":386547056640,"available":386250080256,"internally_reserved":0,"allocated":161546240,"data_stored":323092500,"data_compressed":16664319,"data_compressed_allocated":160649216,"data_compressed_original":322772992,"omap_allocated":0,"internal_metadata":135397376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16351232,"num_objects":250,"num_object_clones":0,"num_object_copies":750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1996,"num_write_kb":15968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002193"},"pg_stats":[{"pgid":"3.f","version":"25'674","reported_seq":707,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:46.551888+0000","last_change":"2024-04-23T01:45:16.856566+0000","last_active":"2024-04-23T01:46:46.551888+0000","last_peered":"2024-04-23T01:46:46.551888+0000","last_clean":"2024-04-23T01:46:46.551888+0000","last_became_active":"2024-04-23T01:45:16.848194+0000","last_became_peered":"2024-04-23T01:45:16.848194+0000","last_unstale":"2024-04-23T01:46:46.551888+0000","last_undegraded":"2024-04-23T01:46:46.551888+0000","last_fullsized":"2024-04-23T01:46:46.551888+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5529600,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":675,"num_write_kb":5400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"25'783","reported_seq":820,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:46.552794+0000","last_change":"2024-04-23T01:45:16.906007+0000","last_active":"2024-04-23T01:46:46.552794+0000","last_peered":"2024-04-23T01:46:46.552794+0000","last_clean":"2024-04-23T01:46:46.552794+0000","last_became_active":"2024-04-23T01:45:16.901215+0000","last_became_peered":"2024-04-23T01:45:16.901215+0000","last_unstale":"2024-04-23T01:46:46.552794+0000","last_undegraded":"2024-04-23T01:46:46.552794+0000","last_fullsized":"2024-04-23T01:46:46.552794+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6422528,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"25'759","reported_seq":792,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:45.955070+0000","last_change":"2024-04-23T01:45:16.886628+0000","last_active":"2024-04-23T01:46:45.955070+0000","last_peered":"2024-04-23T01:46:45.955070+0000","last_clean":"2024-04-23T01:46:45.955070+0000","last_became_active":"2024-04-23T01:45:16.880681+0000","last_became_peered":"2024-04-23T01:45:16.880681+0000","last_unstale":"2024-04-23T01:46:45.955070+0000","last_undegraded":"2024-04-23T01:46:45.955070+0000","last_fullsized":"2024-04-23T01:46:45.955070+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6225920,"num_objects":95,"num_object_clones":0,"num_object_copies":285,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":760,"num_write_kb":6080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"25'879","reported_seq":912,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:45.697352+0000","last_change":"2024-04-23T01:45:16.836330+0000","last_active":"2024-04-23T01:46:45.697352+0000","last_peered":"2024-04-23T01:46:45.697352+0000","last_clean":"2024-04-23T01:46:45.697352+0000","last_became_active":"2024-04-23T01:45:16.821451+0000","last_became_peered":"2024-04-23T01:45:16.821451+0000","last_unstale":"2024-04-23T01:46:45.697352+0000","last_undegraded":"2024-04-23T01:46:45.697352+0000","last_fullsized":"2024-04-23T01:46:45.697352+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7208960,"num_objects":110,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":880,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"25'751","reported_seq":784,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:46.133210+0000","last_change":"2024-04-23T01:45:16.860867+0000","last_active":"2024-04-23T01:46:46.133210+0000","last_peered":"2024-04-23T01:46:46.133210+0000","last_clean":"2024-04-23T01:46:46.133210+0000","last_became_active":"2024-04-23T01:45:16.852271+0000","last_became_peered":"2024-04-23T01:45:16.852271+0000","last_unstale":"2024-04-23T01:46:46.133210+0000","last_undegraded":"2024-04-23T01:46:46.133210+0000","last_fullsized":"2024-04-23T01:46:46.133210+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"25'815","reported_seq":848,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:46.474370+0000","last_change":"2024-04-23T01:45:16.805452+0000","last_active":"2024-04-23T01:46:46.474370+0000","last_peered":"2024-04-23T01:46:46.474370+0000","last_clean":"2024-04-23T01:46:46.474370+0000","last_became_active":"2024-04-23T01:45:16.797805+0000","last_became_peered":"2024-04-23T01:45:16.797805+0000","last_unstale":"2024-04-23T01:46:46.474370+0000","last_undegraded":"2024-04-23T01:46:46.474370+0000","last_fullsized":"2024-04-23T01:46:46.474370+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6684672,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":816,"num_write_kb":6528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"25'895","reported_seq":928,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:46.280267+0000","last_change":"2024-04-23T01:45:16.893894+0000","last_active":"2024-04-23T01:46:46.280267+0000","last_peered":"2024-04-23T01:46:46.280267+0000","last_clean":"2024-04-23T01:46:46.280267+0000","last_became_active":"2024-04-23T01:45:16.888673+0000","last_became_peered":"2024-04-23T01:45:16.888673+0000","last_unstale":"2024-04-23T01:46:46.280267+0000","last_undegraded":"2024-04-23T01:46:46.280267+0000","last_fullsized":"2024-04-23T01:46:46.280267+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340032,"num_objects":112,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":896,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"25'871","reported_seq":904,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:46.367385+0000","last_change":"2024-04-23T01:45:16.889162+0000","last_active":"2024-04-23T01:46:46.367385+0000","last_peered":"2024-04-23T01:46:46.367385+0000","last_clean":"2024-04-23T01:46:46.367385+0000","last_became_active":"2024-04-23T01:45:16.880761+0000","last_became_peered":"2024-04-23T01:45:16.880761+0000","last_unstale":"2024-04-23T01:46:46.367385+0000","last_undegraded":"2024-04-23T01:46:46.367385+0000","last_fullsized":"2024-04-23T01:46:46.367385+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7143424,"num_objects":109,"num_object_clones":0,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":872,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":174,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.777638+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:46:14.777638+0000","last_peered":"2024-04-23T01:46:14.777638+0000","last_clean":"2024-04-23T01:46:14.777638+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:46:14.777638+0000","last_undegraded":"2024-04-23T01:46:14.777638+0000","last_fullsized":"2024-04-23T01:46:14.777638+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.815521+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:46:14.815521+0000","last_peered":"2024-04-23T01:46:14.815521+0000","last_clean":"2024-04-23T01:46:14.815521+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:46:14.815521+0000","last_undegraded":"2024-04-23T01:46:14.815521+0000","last_fullsized":"2024-04-23T01:46:14.815521+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"25'856","reported_seq":889,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:46.286910+0000","last_change":"2024-04-23T01:45:16.891787+0000","last_active":"2024-04-23T01:46:46.286910+0000","last_peered":"2024-04-23T01:46:46.286910+0000","last_clean":"2024-04-23T01:46:46.286910+0000","last_became_active":"2024-04-23T01:45:16.885404+0000","last_became_peered":"2024-04-23T01:45:16.885404+0000","last_unstale":"2024-04-23T01:46:46.286910+0000","last_undegraded":"2024-04-23T01:46:46.286910+0000","last_fullsized":"2024-04-23T01:46:46.286910+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7020544,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":857,"num_write_kb":6856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.256426+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:46:14.256426+0000","last_peered":"2024-04-23T01:46:14.256426+0000","last_clean":"2024-04-23T01:46:14.256426+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:46:14.256426+0000","last_undegraded":"2024-04-23T01:46:14.256426+0000","last_fullsized":"2024-04-23T01:46:14.256426+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"25'868","reported_seq":901,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:46.551056+0000","last_change":"2024-04-23T01:45:16.919259+0000","last_active":"2024-04-23T01:46:46.551056+0000","last_peered":"2024-04-23T01:46:46.551056+0000","last_clean":"2024-04-23T01:46:46.551056+0000","last_became_active":"2024-04-23T01:45:16.912983+0000","last_became_peered":"2024-04-23T01:45:16.912983+0000","last_unstale":"2024-04-23T01:46:46.551056+0000","last_undegraded":"2024-04-23T01:46:46.551056+0000","last_fullsized":"2024-04-23T01:46:46.551056+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":868,"log_dups_size":0,"ondisk_log_size":868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7118848,"num_objects":109,"num_object_clones":0,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":869,"num_write_kb":6952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.788785+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:46:14.788785+0000","last_peered":"2024-04-23T01:46:14.788785+0000","last_clean":"2024-04-23T01:46:14.788785+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:46:14.788785+0000","last_undegraded":"2024-04-23T01:46:14.788785+0000","last_fullsized":"2024-04-23T01:46:14.788785+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"25'719","reported_seq":752,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:46.563452+0000","last_change":"2024-04-23T01:45:16.897089+0000","last_active":"2024-04-23T01:46:46.563452+0000","last_peered":"2024-04-23T01:46:46.563452+0000","last_clean":"2024-04-23T01:46:46.563452+0000","last_became_active":"2024-04-23T01:45:16.891776+0000","last_became_peered":"2024-04-23T01:45:16.891776+0000","last_unstale":"2024-04-23T01:46:46.563452+0000","last_undegraded":"2024-04-23T01:46:46.563452+0000","last_fullsized":"2024-04-23T01:46:46.563452+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":720,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.263415+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:46:14.263415+0000","last_peered":"2024-04-23T01:46:14.263415+0000","last_clean":"2024-04-23T01:46:14.263415+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:46:14.263415+0000","last_undegraded":"2024-04-23T01:46:14.263415+0000","last_fullsized":"2024-04-23T01:46:14.263415+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"25'903","reported_seq":940,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:44.696376+0000","last_change":"2024-04-23T01:45:16.823909+0000","last_active":"2024-04-23T01:46:44.696376+0000","last_peered":"2024-04-23T01:46:44.696376+0000","last_clean":"2024-04-23T01:46:44.696376+0000","last_became_active":"2024-04-23T01:45:16.815467+0000","last_became_peered":"2024-04-23T01:45:16.815467+0000","last_unstale":"2024-04-23T01:46:44.696376+0000","last_undegraded":"2024-04-23T01:46:44.696376+0000","last_fullsized":"2024-04-23T01:46:44.696376+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":903,"log_dups_size":0,"ondisk_log_size":903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7405568,"num_objects":113,"num_object_clones":0,"num_object_copies":339,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":904,"num_write_kb":7232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"25'871","reported_seq":904,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:44.260819+0000","last_change":"2024-04-23T01:45:16.859980+0000","last_active":"2024-04-23T01:46:44.260819+0000","last_peered":"2024-04-23T01:46:44.260819+0000","last_clean":"2024-04-23T01:46:44.260819+0000","last_became_active":"2024-04-23T01:45:16.855622+0000","last_became_peered":"2024-04-23T01:45:16.855622+0000","last_unstale":"2024-04-23T01:46:44.260819+0000","last_undegraded":"2024-04-23T01:46:44.260819+0000","last_fullsized":"2024-04-23T01:46:44.260819+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7143424,"num_objects":109,"num_object_clones":0,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":872,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.805143+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:46:14.805143+0000","last_peered":"2024-04-23T01:46:14.805143+0000","last_clean":"2024-04-23T01:46:14.805143+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:46:14.805143+0000","last_undegraded":"2024-04-23T01:46:14.805143+0000","last_fullsized":"2024-04-23T01:46:14.805143+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"25'687","reported_seq":724,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:45.608982+0000","last_change":"2024-04-23T01:45:16.854641+0000","last_active":"2024-04-23T01:46:45.608982+0000","last_peered":"2024-04-23T01:46:45.608982+0000","last_clean":"2024-04-23T01:46:45.608982+0000","last_became_active":"2024-04-23T01:45:16.845783+0000","last_became_peered":"2024-04-23T01:45:16.845783+0000","last_unstale":"2024-04-23T01:46:45.608982+0000","last_undegraded":"2024-04-23T01:46:45.608982+0000","last_fullsized":"2024-04-23T01:46:45.608982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5636096,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":688,"num_write_kb":5504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.802955+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:46:14.802955+0000","last_peered":"2024-04-23T01:46:14.802955+0000","last_clean":"2024-04-23T01:46:14.802955+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:46:14.802955+0000","last_undegraded":"2024-04-23T01:46:14.802955+0000","last_fullsized":"2024-04-23T01:46:14.802955+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"25'991","reported_seq":1024,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:45.196965+0000","last_change":"2024-04-23T01:45:16.845162+0000","last_active":"2024-04-23T01:46:45.196965+0000","last_peered":"2024-04-23T01:46:45.196965+0000","last_clean":"2024-04-23T01:46:45.196965+0000","last_became_active":"2024-04-23T01:45:16.829836+0000","last_became_peered":"2024-04-23T01:45:16.829836+0000","last_unstale":"2024-04-23T01:46:45.196965+0000","last_undegraded":"2024-04-23T01:46:45.196965+0000","last_fullsized":"2024-04-23T01:46:45.196965+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":991,"log_dups_size":0,"ondisk_log_size":991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8126464,"num_objects":124,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":992,"num_write_kb":7936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":43,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.793283+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:46:14.793283+0000","last_peered":"2024-04-23T01:46:14.793283+0000","last_clean":"2024-04-23T01:46:14.793283+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:46:14.793283+0000","last_undegraded":"2024-04-23T01:46:14.793283+0000","last_fullsized":"2024-04-23T01:46:14.793283+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"25'847","reported_seq":884,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:45.994453+0000","last_change":"2024-04-23T01:45:16.885391+0000","last_active":"2024-04-23T01:46:45.994453+0000","last_peered":"2024-04-23T01:46:45.994453+0000","last_clean":"2024-04-23T01:46:45.994453+0000","last_became_active":"2024-04-23T01:45:16.880317+0000","last_became_peered":"2024-04-23T01:45:16.880317+0000","last_unstale":"2024-04-23T01:46:45.994453+0000","last_undegraded":"2024-04-23T01:46:45.994453+0000","last_fullsized":"2024-04-23T01:46:45.994453+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6946816,"num_objects":106,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":848,"num_write_kb":6784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.803589+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:46:14.803589+0000","last_peered":"2024-04-23T01:46:14.803589+0000","last_clean":"2024-04-23T01:46:14.803589+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:46:14.803589+0000","last_undegraded":"2024-04-23T01:46:14.803589+0000","last_fullsized":"2024-04-23T01:46:14.803589+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":108011520,"num_objects":1650,"num_object_clones":0,"num_object_copies":4950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13185,"num_write_kb":105480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":162000896,"data_stored":324001792,"data_compressed":16726740,"data_compressed_allocated":162000896,"data_compressed_original":324001792,"omap_allocated":0,"internal_metadata":0},"log_size":13169,"ondisk_log_size":13169,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705763,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":85128,"kb_used_data":45824,"kb_used_omap":0,"kb_used_meta":39296,"kb_avail":94286712,"statfs":{"total":96636764160,"available":96549593088,"internally_reserved":0,"allocated":46923776,"data_stored":93964429,"data_compressed":4847004,"data_compressed_allocated":46694400,"data_compressed_original":93880320,"omap_allocated":0,"internal_metadata":40239104},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705763,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":67004,"kb_used_data":36020,"kb_used_omap":0,"kb_used_meta":30976,"kb_avail":94304836,"statfs":{"total":96636764160,"available":96568152064,"internally_reserved":0,"allocated":36884480,"data_stored":73885837,"data_compressed":3810420,"data_compressed_allocated":36655104,"data_compressed_original":73801728,"omap_allocated":0,"internal_metadata":31719424},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705763,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":70576,"kb_used_data":39592,"kb_used_omap":0,"kb_used_meta":30976,"kb_avail":94301264,"statfs":{"total":96636764160,"available":96564494336,"internally_reserved":0,"allocated":40542208,"data_stored":80733805,"data_compressed":4164362,"data_compressed_allocated":40333312,"data_compressed_original":80666624,"omap_allocated":0,"internal_metadata":31719424},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705763,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":67308,"kb_used_data":36324,"kb_used_omap":0,"kb_used_meta":30976,"kb_avail":94304532,"statfs":{"total":96636764160,"available":96567840768,"internally_reserved":0,"allocated":37195776,"data_stored":74508429,"data_compressed":3842533,"data_compressed_allocated":36966400,"data_compressed_original":74424320,"omap_allocated":0,"internal_metadata":31719424},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":37220352,"data_stored":74440704,"data_compressed":3843036,"data_compressed_allocated":37220352,"data_compressed_original":74440704,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40800256,"data_stored":81600512,"data_compressed":4212584,"data_compressed_allocated":40800256,"data_compressed_original":81600512,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36995072,"data_stored":73990144,"data_compressed":3819806,"data_compressed_allocated":36995072,"data_compressed_original":73990144,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":46985216,"data_stored":93970432,"data_compressed":4851314,"data_compressed_allocated":46985216,"data_compressed_original":93970432,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:46:50.391 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:46:50.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:50.521+0000 7fee1dc6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:50.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:50.530+0000 7fee1dc6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:52.940 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:46:53.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:53.088+0000 7f0b6939d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:53.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:53.097+0000 7f0b6939d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:54.413 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:46:54.414 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:46:54.414 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:46:54.414 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:46:54.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:54.545+0000 7f8221a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:54.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:54.562+0000 7f8221a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:54.595 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:46:54.595 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:46:54.595 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:46:54.595 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:46:54.595 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:46:54.595 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:46:54.595 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:46:54.595 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:46:54.595 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:46:54.595 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:46:54.596 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:46:54.596 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:46:54.598 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:46:54.598 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:46:54.599 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:46:54.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:54.742+0000 7f55fe39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:54.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:54.756+0000 7f55fe39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:54.803 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:46:54.803 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:46:54.803 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:46:54.804 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:46:54.804 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:46:54.804 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:46:54.804 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:46:54.804 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:46:54.804 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:46:54.804 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:46:54.804 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:46:54.804 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:46:54.807 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:46:54.807 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:46:54.808 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:46:54.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:54.944+0000 7f851ed5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:54.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:54.962+0000 7f851ed5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:55.002 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:46:55.002 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:46:55.002 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:46:55.002 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:46:55.002 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:46:55.003 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:46:55.003 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:46:55.003 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:46:55.003 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:46:55.003 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:46:55.003 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:46:55.003 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:46:55.006 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:46:55.007 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:46:55.007 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:46:55.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:55.147+0000 7f2fecaed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:55.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:55.163+0000 7f2fecaed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:55.303 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:46:55.304 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:46:55.304 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:46:55.304 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:46:55.304 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:46:55.304 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:46:55.304 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:46:55.304 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:46:55.304 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:46:55.304 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:46:55.304 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:46:55.304 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:46:55.307 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:46:55.307 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:46:55.540 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:46:55.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:55.692+0000 7efe1c2ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:55.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:55.700+0000 7efe1c2ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:58.124 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:46:58.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:58.251+0000 7ff00186d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:58.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:46:58.259+0000 7ff00186d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:46:58.307 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Removed 1671 objects 2024-04-23T01:46:58.307 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Clean up completed and total clean up time :9.4874 2024-04-23T01:47:00.308 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:47:00.308 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:47:00.309 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:47:00.372 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:47:00.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:00.431+0000 7f6fac26d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:00.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:00.439+0000 7f6fac26d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:00.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:00.486+0000 7f467155d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:00.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:00.491+0000 7f467155d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:00.636 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:47:00.636 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:47:00.647 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-04-23T01:46:59.569600+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19735096,"num_objects":296,"num_object_clones":0,"num_object_copies":888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":14877,"num_write_kb":108783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14821,"ondisk_log_size":14821,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":176732,"kb_used_data":44380,"kb_used_omap":0,"kb_used_meta":132224,"kb_avail":377310628,"statfs":{"total":386547056640,"available":386366083072,"internally_reserved":0,"allocated":45445120,"data_stored":90865756,"data_compressed":4676445,"data_compressed_allocated":44535808,"data_compressed_original":90546176,"omap_allocated":0,"internal_metadata":135397376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-88866792,"num_objects":-1356,"num_object_clones":0,"num_object_copies":-4068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1555,"num_write_kb":1409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002626"},"pg_stats":[{"pgid":"3.f","version":"25'768","reported_seq":801,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:56.539300+0000","last_change":"2024-04-23T01:45:16.856566+0000","last_active":"2024-04-23T01:46:56.539300+0000","last_peered":"2024-04-23T01:46:56.539300+0000","last_clean":"2024-04-23T01:46:56.539300+0000","last_became_active":"2024-04-23T01:45:16.848194+0000","last_became_peered":"2024-04-23T01:45:16.848194+0000","last_unstale":"2024-04-23T01:46:56.539300+0000","last_undegraded":"2024-04-23T01:46:56.539300+0000","last_fullsized":"2024-04-23T01:46:56.539300+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":768,"log_dups_size":0,"ondisk_log_size":768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":769,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"25'879","reported_seq":916,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:56.215948+0000","last_change":"2024-04-23T01:45:16.906007+0000","last_active":"2024-04-23T01:46:56.215948+0000","last_peered":"2024-04-23T01:46:56.215948+0000","last_clean":"2024-04-23T01:46:56.215948+0000","last_became_active":"2024-04-23T01:45:16.901215+0000","last_became_peered":"2024-04-23T01:45:16.901215+0000","last_unstale":"2024-04-23T01:46:56.215948+0000","last_undegraded":"2024-04-23T01:46:56.215948+0000","last_fullsized":"2024-04-23T01:46:56.215948+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":720896,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":880,"num_write_kb":6336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"25'849","reported_seq":883,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:56.310320+0000","last_change":"2024-04-23T01:45:16.886628+0000","last_active":"2024-04-23T01:46:56.310320+0000","last_peered":"2024-04-23T01:46:56.310320+0000","last_clean":"2024-04-23T01:46:56.310320+0000","last_became_active":"2024-04-23T01:45:16.880681+0000","last_became_peered":"2024-04-23T01:45:16.880681+0000","last_unstale":"2024-04-23T01:46:56.310320+0000","last_undegraded":"2024-04-23T01:46:56.310320+0000","last_fullsized":"2024-04-23T01:46:56.310320+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":849,"log_dups_size":0,"ondisk_log_size":849,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983064,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":850,"num_write_kb":6145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"25'979","reported_seq":1012,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:56.335009+0000","last_change":"2024-04-23T01:45:16.836330+0000","last_active":"2024-04-23T01:46:56.335009+0000","last_peered":"2024-04-23T01:46:56.335009+0000","last_clean":"2024-04-23T01:46:56.335009+0000","last_became_active":"2024-04-23T01:45:16.821451+0000","last_became_peered":"2024-04-23T01:45:16.821451+0000","last_unstale":"2024-04-23T01:46:56.335009+0000","last_undegraded":"2024-04-23T01:46:56.335009+0000","last_fullsized":"2024-04-23T01:46:56.335009+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":979,"log_dups_size":0,"ondisk_log_size":979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":980,"num_write_kb":7104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"25'842","reported_seq":875,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:56.194743+0000","last_change":"2024-04-23T01:45:16.860867+0000","last_active":"2024-04-23T01:46:56.194743+0000","last_peered":"2024-04-23T01:46:56.194743+0000","last_clean":"2024-04-23T01:46:56.194743+0000","last_became_active":"2024-04-23T01:45:16.852271+0000","last_became_peered":"2024-04-23T01:45:16.852271+0000","last_unstale":"2024-04-23T01:46:56.194743+0000","last_undegraded":"2024-04-23T01:46:56.194743+0000","last_fullsized":"2024-04-23T01:46:56.194743+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":842,"log_dups_size":0,"ondisk_log_size":842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":843,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"25'909","reported_seq":942,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:56.550262+0000","last_change":"2024-04-23T01:45:16.805452+0000","last_active":"2024-04-23T01:46:56.550262+0000","last_peered":"2024-04-23T01:46:56.550262+0000","last_clean":"2024-04-23T01:46:56.550262+0000","last_became_active":"2024-04-23T01:45:16.797805+0000","last_became_peered":"2024-04-23T01:45:16.797805+0000","last_unstale":"2024-04-23T01:46:56.550262+0000","last_undegraded":"2024-04-23T01:46:56.550262+0000","last_fullsized":"2024-04-23T01:46:56.550262+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":909,"log_dups_size":0,"ondisk_log_size":909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":910,"num_write_kb":6592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"25'999","reported_seq":1032,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:56.396469+0000","last_change":"2024-04-23T01:45:16.893894+0000","last_active":"2024-04-23T01:46:56.396469+0000","last_peered":"2024-04-23T01:46:56.396469+0000","last_clean":"2024-04-23T01:46:56.396469+0000","last_became_active":"2024-04-23T01:45:16.888673+0000","last_became_peered":"2024-04-23T01:45:16.888673+0000","last_unstale":"2024-04-23T01:46:56.396469+0000","last_undegraded":"2024-04-23T01:46:56.396469+0000","last_fullsized":"2024-04-23T01:46:56.396469+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":999,"log_dups_size":0,"ondisk_log_size":999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1000,"num_write_kb":7232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"25'978","reported_seq":1011,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:56.394237+0000","last_change":"2024-04-23T01:45:16.889162+0000","last_active":"2024-04-23T01:46:56.394237+0000","last_peered":"2024-04-23T01:46:56.394237+0000","last_clean":"2024-04-23T01:46:56.394237+0000","last_became_active":"2024-04-23T01:45:16.880761+0000","last_became_peered":"2024-04-23T01:45:16.880761+0000","last_unstale":"2024-04-23T01:46:56.394237+0000","last_undegraded":"2024-04-23T01:46:56.394237+0000","last_fullsized":"2024-04-23T01:46:56.394237+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":978,"log_dups_size":0,"ondisk_log_size":978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310720,"num_objects":20,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":979,"num_write_kb":7104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":174,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.777638+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:46:14.777638+0000","last_peered":"2024-04-23T01:46:14.777638+0000","last_clean":"2024-04-23T01:46:14.777638+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:46:14.777638+0000","last_undegraded":"2024-04-23T01:46:14.777638+0000","last_fullsized":"2024-04-23T01:46:14.777638+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.815521+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:46:14.815521+0000","last_peered":"2024-04-23T01:46:14.815521+0000","last_clean":"2024-04-23T01:46:14.815521+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:46:14.815521+0000","last_undegraded":"2024-04-23T01:46:14.815521+0000","last_fullsized":"2024-04-23T01:46:14.815521+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"25'962","reported_seq":995,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:56.490565+0000","last_change":"2024-04-23T01:45:16.891787+0000","last_active":"2024-04-23T01:46:56.490565+0000","last_peered":"2024-04-23T01:46:56.490565+0000","last_clean":"2024-04-23T01:46:56.490565+0000","last_became_active":"2024-04-23T01:45:16.885404+0000","last_became_peered":"2024-04-23T01:45:16.885404+0000","last_unstale":"2024-04-23T01:46:56.490565+0000","last_undegraded":"2024-04-23T01:46:56.490565+0000","last_fullsized":"2024-04-23T01:46:56.490565+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":962,"log_dups_size":0,"ondisk_log_size":962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1769472,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":963,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.256426+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:46:14.256426+0000","last_peered":"2024-04-23T01:46:14.256426+0000","last_clean":"2024-04-23T01:46:14.256426+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:46:14.256426+0000","last_undegraded":"2024-04-23T01:46:14.256426+0000","last_fullsized":"2024-04-23T01:46:14.256426+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"25'982","reported_seq":1015,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:56.394870+0000","last_change":"2024-04-23T01:45:16.919259+0000","last_active":"2024-04-23T01:46:56.394870+0000","last_peered":"2024-04-23T01:46:56.394870+0000","last_clean":"2024-04-23T01:46:56.394870+0000","last_became_active":"2024-04-23T01:45:16.912983+0000","last_became_peered":"2024-04-23T01:45:16.912983+0000","last_unstale":"2024-04-23T01:46:56.394870+0000","last_undegraded":"2024-04-23T01:46:56.394870+0000","last_fullsized":"2024-04-23T01:46:56.394870+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":983,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.788785+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:46:14.788785+0000","last_peered":"2024-04-23T01:46:14.788785+0000","last_clean":"2024-04-23T01:46:14.788785+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:46:14.788785+0000","last_undegraded":"2024-04-23T01:46:14.788785+0000","last_fullsized":"2024-04-23T01:46:14.788785+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"25'813","reported_seq":846,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:56.402801+0000","last_change":"2024-04-23T01:45:16.897089+0000","last_active":"2024-04-23T01:46:56.402801+0000","last_peered":"2024-04-23T01:46:56.402801+0000","last_clean":"2024-04-23T01:46:56.402801+0000","last_became_active":"2024-04-23T01:45:16.891776+0000","last_became_peered":"2024-04-23T01:45:16.891776+0000","last_unstale":"2024-04-23T01:46:56.402801+0000","last_undegraded":"2024-04-23T01:46:56.402801+0000","last_fullsized":"2024-04-23T01:46:56.402801+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":813,"log_dups_size":0,"ondisk_log_size":813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":814,"num_write_kb":5888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.263415+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:46:14.263415+0000","last_peered":"2024-04-23T01:46:14.263415+0000","last_clean":"2024-04-23T01:46:14.263415+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:46:14.263415+0000","last_undegraded":"2024-04-23T01:46:14.263415+0000","last_fullsized":"2024-04-23T01:46:14.263415+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"25'1002","reported_seq":1039,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:56.440299+0000","last_change":"2024-04-23T01:45:16.823909+0000","last_active":"2024-04-23T01:46:56.440299+0000","last_peered":"2024-04-23T01:46:56.440299+0000","last_clean":"2024-04-23T01:46:56.440299+0000","last_became_active":"2024-04-23T01:45:16.815467+0000","last_became_peered":"2024-04-23T01:45:16.815467+0000","last_unstale":"2024-04-23T01:46:56.440299+0000","last_undegraded":"2024-04-23T01:46:56.440299+0000","last_fullsized":"2024-04-23T01:46:56.440299+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":1002,"log_dups_size":0,"ondisk_log_size":1002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1003,"num_write_kb":7232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"25'960","reported_seq":993,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:56.232855+0000","last_change":"2024-04-23T01:45:16.859980+0000","last_active":"2024-04-23T01:46:56.232855+0000","last_peered":"2024-04-23T01:46:56.232855+0000","last_clean":"2024-04-23T01:46:56.232855+0000","last_became_active":"2024-04-23T01:45:16.855622+0000","last_became_peered":"2024-04-23T01:45:16.855622+0000","last_unstale":"2024-04-23T01:46:56.232855+0000","last_undegraded":"2024-04-23T01:46:56.232855+0000","last_fullsized":"2024-04-23T01:46:56.232855+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":960,"log_dups_size":0,"ondisk_log_size":960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310720,"num_objects":20,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":961,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.805143+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:46:14.805143+0000","last_peered":"2024-04-23T01:46:14.805143+0000","last_clean":"2024-04-23T01:46:14.805143+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:46:14.805143+0000","last_undegraded":"2024-04-23T01:46:14.805143+0000","last_fullsized":"2024-04-23T01:46:14.805143+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"25'767","reported_seq":804,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:56.360850+0000","last_change":"2024-04-23T01:45:16.854641+0000","last_active":"2024-04-23T01:46:56.360850+0000","last_peered":"2024-04-23T01:46:56.360850+0000","last_clean":"2024-04-23T01:46:56.360850+0000","last_became_active":"2024-04-23T01:45:16.845783+0000","last_became_peered":"2024-04-23T01:45:16.845783+0000","last_unstale":"2024-04-23T01:46:56.360850+0000","last_undegraded":"2024-04-23T01:46:56.360850+0000","last_fullsized":"2024-04-23T01:46:56.360850+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1646592,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":768,"num_write_kb":5640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.802955+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:46:14.802955+0000","last_peered":"2024-04-23T01:46:14.802955+0000","last_clean":"2024-04-23T01:46:14.802955+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:46:14.802955+0000","last_undegraded":"2024-04-23T01:46:14.802955+0000","last_fullsized":"2024-04-23T01:46:14.802955+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"25'1098","reported_seq":1131,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:56.521752+0000","last_change":"2024-04-23T01:45:16.845162+0000","last_active":"2024-04-23T01:46:56.521752+0000","last_peered":"2024-04-23T01:46:56.521752+0000","last_clean":"2024-04-23T01:46:56.521752+0000","last_became_active":"2024-04-23T01:45:16.829836+0000","last_became_peered":"2024-04-23T01:45:16.829836+0000","last_unstale":"2024-04-23T01:46:56.521752+0000","last_undegraded":"2024-04-23T01:46:56.521752+0000","last_fullsized":"2024-04-23T01:46:56.521752+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":1098,"log_dups_size":0,"ondisk_log_size":1098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1099,"num_write_kb":7936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":43,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.793283+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:46:14.793283+0000","last_peered":"2024-04-23T01:46:14.793283+0000","last_clean":"2024-04-23T01:46:14.793283+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:46:14.793283+0000","last_undegraded":"2024-04-23T01:46:14.793283+0000","last_fullsized":"2024-04-23T01:46:14.793283+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"25'937","reported_seq":974,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:56.240576+0000","last_change":"2024-04-23T01:45:16.885391+0000","last_active":"2024-04-23T01:46:56.240576+0000","last_peered":"2024-04-23T01:46:56.240576+0000","last_clean":"2024-04-23T01:46:56.240576+0000","last_became_active":"2024-04-23T01:45:16.880317+0000","last_became_peered":"2024-04-23T01:45:16.880317+0000","last_unstale":"2024-04-23T01:46:56.240576+0000","last_undegraded":"2024-04-23T01:46:56.240576+0000","last_fullsized":"2024-04-23T01:46:56.240576+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:15.698350+0000","last_clean_scrub_stamp":"2024-04-23T01:45:15.698350+0000","objects_scrubbed":0,"log_size":937,"log_dups_size":0,"ondisk_log_size":937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":938,"num_write_kb":6784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.803589+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:46:14.803589+0000","last_peered":"2024-04-23T01:46:14.803589+0000","last_clean":"2024-04-23T01:46:14.803589+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:46:14.803589+0000","last_undegraded":"2024-04-23T01:46:14.803589+0000","last_fullsized":"2024-04-23T01:46:14.803589+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":19144728,"num_objects":294,"num_object_clones":0,"num_object_copies":882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14740,"num_write_kb":106889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":29024256,"data_stored":58024008,"data_compressed":2996106,"data_compressed_allocated":29011968,"data_compressed_original":58023936,"omap_allocated":0,"internal_metadata":0},"log_size":14724,"ondisk_log_size":14724,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705773,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":39864,"kb_used_data":13008,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94331976,"statfs":{"total":96636764160,"available":96595943424,"internally_reserved":0,"allocated":13320192,"data_stored":26749093,"data_compressed":1377207,"data_compressed_allocated":13086720,"data_compressed_original":26664960,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705773,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":44948,"kb_used_data":9772,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94326892,"statfs":{"total":96636764160,"available":96590737408,"internally_reserved":0,"allocated":10006528,"data_stored":20129933,"data_compressed":1035424,"data_compressed_allocated":9777152,"data_compressed_original":20045824,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705773,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46264,"kb_used_data":11088,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94325576,"statfs":{"total":96636764160,"available":96589389824,"internally_reserved":0,"allocated":11354112,"data_stored":22349445,"data_compressed":1150559,"data_compressed_allocated":11141120,"data_compressed_original":22282240,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705773,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":45656,"kb_used_data":10512,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94326184,"statfs":{"total":96636764160,"available":96590012416,"internally_reserved":0,"allocated":10764288,"data_stored":21637285,"data_compressed":1113255,"data_compressed_allocated":10530816,"data_compressed_original":21553152,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7086080,"data_stored":14163992,"data_compressed":731366,"data_compressed_allocated":7081984,"data_compressed_original":14163968,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6983680,"data_stored":13959192,"data_compressed":720791,"data_compressed_allocated":6979584,"data_compressed_original":13959168,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6393856,"data_stored":12787712,"data_compressed":660303,"data_compressed_allocated":6393856,"data_compressed_original":12787712,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8560640,"data_stored":17113112,"data_compressed":883646,"data_compressed_allocated":8556544,"data_compressed_original":17113088,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:47:01.451 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_0 2024-04-23T01:47:01.452 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-04-23T01:47:01.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:01.561+0000 7ffb7dc6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:01.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:01.573+0000 7ffb7dc6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:02.283 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'unique_pool_0' removed 2024-04-23T01:47:02.298 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-04-23T01:47:02.298 INFO:tasks.radosbench:Beginning radosbench... 2024-04-23T01:47:02.299 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_1 2024-04-23T01:47:02.299 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_1 16 2024-04-23T01:47:02.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:02.404+0000 7f2f6039d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:02.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:02.416+0000 7f2f6039d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:02.587 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:47:02.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:02.690+0000 7f36634ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:02.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:02.696+0000 7f36634ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:03.292 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'unique_pool_1' created 2024-04-23T01:47:03.308 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_1 rados --yes-i-really-mean-it || true 2024-04-23T01:47:03.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:03.440+0000 7f3df946d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:03.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:03.455+0000 7f3df946d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:04.298 INFO:teuthology.orchestra.run.smithi090.stderr:enabled application 'rados' on pool 'unique_pool_1' 2024-04-23T01:47:04.877 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:47:04.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:04.980+0000 7f5584020640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:04.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:04.986+0000 7f5584020640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:05.317 DEBUG:teuthology.orchestra.run.smithi090:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 8192 --object-size 65536 -p unique_pool_1 bench 90 write' 2024-04-23T01:47:05.320 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-04-23T01:47:05.320 INFO:teuthology.orchestra.run:waiting for 3000 2024-04-23T01:47:05.380 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:hints = 1 2024-04-23T01:47:05.380 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Maintaining 16 concurrent writes of 8192 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-04-23T01:47:05.381 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Object prefix: benchmark_data_smithi090_33413 2024-04-23T01:47:05.381 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:47:05.381 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 0 0 0 0 0 0 - 0 2024-04-23T01:47:05.650 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:47:05.650 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:47:05.650 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:47:05.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:05.783+0000 7f2ed8e64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:05.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:05.793+0000 7f2ed8e64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:06.037 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:47:06.037 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:47:06.052 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-04-23T01:47:05.570940+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":141572,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":140544,"kb_avail":377345788,"statfs":{"total":386547056640,"available":386402086912,"internally_reserved":0,"allocated":1019904,"data_stored":2039828,"data_compressed":89859,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":143917056},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19144728,"num_objects":-294,"num_object_clones":0,"num_object_copies":-882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-14740,"num_write_kb":-106889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000849"},"pg_stats":[{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:47:03.285808+0000","last_change":"2024-04-23T01:47:03.285808+0000","last_active":"2024-04-23T01:47:03.285808+0000","last_peered":"2024-04-23T01:47:03.285808+0000","last_clean":"2024-04-23T01:47:03.285808+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:47:03.285808+0000","last_undegraded":"2024-04-23T01:47:03.285808+0000","last_fullsized":"2024-04-23T01:47:03.285808+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:47:03.285808+0000","last_change":"2024-04-23T01:47:03.285808+0000","last_active":"2024-04-23T01:47:03.285808+0000","last_peered":"2024-04-23T01:47:03.285808+0000","last_clean":"2024-04-23T01:47:03.285808+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:47:03.285808+0000","last_undegraded":"2024-04-23T01:47:03.285808+0000","last_fullsized":"2024-04-23T01:47:03.285808+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:47:03.285808+0000","last_change":"2024-04-23T01:47:03.285808+0000","last_active":"2024-04-23T01:47:03.285808+0000","last_peered":"2024-04-23T01:47:03.285808+0000","last_clean":"2024-04-23T01:47:03.285808+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:47:03.285808+0000","last_undegraded":"2024-04-23T01:47:03.285808+0000","last_fullsized":"2024-04-23T01:47:03.285808+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:47:03.285808+0000","last_change":"2024-04-23T01:47:03.285808+0000","last_active":"2024-04-23T01:47:03.285808+0000","last_peered":"2024-04-23T01:47:03.285808+0000","last_clean":"2024-04-23T01:47:03.285808+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:47:03.285808+0000","last_undegraded":"2024-04-23T01:47:03.285808+0000","last_fullsized":"2024-04-23T01:47:03.285808+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:47:03.285808+0000","last_change":"2024-04-23T01:47:03.285808+0000","last_active":"2024-04-23T01:47:03.285808+0000","last_peered":"2024-04-23T01:47:03.285808+0000","last_clean":"2024-04-23T01:47:03.285808+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:47:03.285808+0000","last_undegraded":"2024-04-23T01:47:03.285808+0000","last_fullsized":"2024-04-23T01:47:03.285808+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:47:03.285808+0000","last_change":"2024-04-23T01:47:03.285808+0000","last_active":"2024-04-23T01:47:03.285808+0000","last_peered":"2024-04-23T01:47:03.285808+0000","last_clean":"2024-04-23T01:47:03.285808+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:47:03.285808+0000","last_undegraded":"2024-04-23T01:47:03.285808+0000","last_fullsized":"2024-04-23T01:47:03.285808+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:47:03.285808+0000","last_change":"2024-04-23T01:47:03.285808+0000","last_active":"2024-04-23T01:47:03.285808+0000","last_peered":"2024-04-23T01:47:03.285808+0000","last_clean":"2024-04-23T01:47:03.285808+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:47:03.285808+0000","last_undegraded":"2024-04-23T01:47:03.285808+0000","last_fullsized":"2024-04-23T01:47:03.285808+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:47:03.285808+0000","last_change":"2024-04-23T01:47:03.285808+0000","last_active":"2024-04-23T01:47:03.285808+0000","last_peered":"2024-04-23T01:47:03.285808+0000","last_clean":"2024-04-23T01:47:03.285808+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:47:03.285808+0000","last_undegraded":"2024-04-23T01:47:03.285808+0000","last_fullsized":"2024-04-23T01:47:03.285808+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:47:03.285808+0000","last_change":"2024-04-23T01:47:03.285808+0000","last_active":"2024-04-23T01:47:03.285808+0000","last_peered":"2024-04-23T01:47:03.285808+0000","last_clean":"2024-04-23T01:47:03.285808+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:47:03.285808+0000","last_undegraded":"2024-04-23T01:47:03.285808+0000","last_fullsized":"2024-04-23T01:47:03.285808+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":174,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.777638+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:46:14.777638+0000","last_peered":"2024-04-23T01:46:14.777638+0000","last_clean":"2024-04-23T01:46:14.777638+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:46:14.777638+0000","last_undegraded":"2024-04-23T01:46:14.777638+0000","last_fullsized":"2024-04-23T01:46:14.777638+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.815521+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:46:14.815521+0000","last_peered":"2024-04-23T01:46:14.815521+0000","last_clean":"2024-04-23T01:46:14.815521+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:46:14.815521+0000","last_undegraded":"2024-04-23T01:46:14.815521+0000","last_fullsized":"2024-04-23T01:46:14.815521+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:47:03.285808+0000","last_change":"2024-04-23T01:47:03.285808+0000","last_active":"2024-04-23T01:47:03.285808+0000","last_peered":"2024-04-23T01:47:03.285808+0000","last_clean":"2024-04-23T01:47:03.285808+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:47:03.285808+0000","last_undegraded":"2024-04-23T01:47:03.285808+0000","last_fullsized":"2024-04-23T01:47:03.285808+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.256426+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:46:14.256426+0000","last_peered":"2024-04-23T01:46:14.256426+0000","last_clean":"2024-04-23T01:46:14.256426+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:46:14.256426+0000","last_undegraded":"2024-04-23T01:46:14.256426+0000","last_fullsized":"2024-04-23T01:46:14.256426+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:47:03.285808+0000","last_change":"2024-04-23T01:47:03.285808+0000","last_active":"2024-04-23T01:47:03.285808+0000","last_peered":"2024-04-23T01:47:03.285808+0000","last_clean":"2024-04-23T01:47:03.285808+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:47:03.285808+0000","last_undegraded":"2024-04-23T01:47:03.285808+0000","last_fullsized":"2024-04-23T01:47:03.285808+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.788785+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:46:14.788785+0000","last_peered":"2024-04-23T01:46:14.788785+0000","last_clean":"2024-04-23T01:46:14.788785+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:46:14.788785+0000","last_undegraded":"2024-04-23T01:46:14.788785+0000","last_fullsized":"2024-04-23T01:46:14.788785+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:47:03.285808+0000","last_change":"2024-04-23T01:47:03.285808+0000","last_active":"2024-04-23T01:47:03.285808+0000","last_peered":"2024-04-23T01:47:03.285808+0000","last_clean":"2024-04-23T01:47:03.285808+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:47:03.285808+0000","last_undegraded":"2024-04-23T01:47:03.285808+0000","last_fullsized":"2024-04-23T01:47:03.285808+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.263415+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:46:14.263415+0000","last_peered":"2024-04-23T01:46:14.263415+0000","last_clean":"2024-04-23T01:46:14.263415+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:46:14.263415+0000","last_undegraded":"2024-04-23T01:46:14.263415+0000","last_fullsized":"2024-04-23T01:46:14.263415+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:47:03.285808+0000","last_change":"2024-04-23T01:47:03.285808+0000","last_active":"2024-04-23T01:47:03.285808+0000","last_peered":"2024-04-23T01:47:03.285808+0000","last_clean":"2024-04-23T01:47:03.285808+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:47:03.285808+0000","last_undegraded":"2024-04-23T01:47:03.285808+0000","last_fullsized":"2024-04-23T01:47:03.285808+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.805143+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:46:14.805143+0000","last_peered":"2024-04-23T01:46:14.805143+0000","last_clean":"2024-04-23T01:46:14.805143+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:46:14.805143+0000","last_undegraded":"2024-04-23T01:46:14.805143+0000","last_fullsized":"2024-04-23T01:46:14.805143+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:47:03.285808+0000","last_change":"2024-04-23T01:47:03.285808+0000","last_active":"2024-04-23T01:47:03.285808+0000","last_peered":"2024-04-23T01:47:03.285808+0000","last_clean":"2024-04-23T01:47:03.285808+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:47:03.285808+0000","last_undegraded":"2024-04-23T01:47:03.285808+0000","last_fullsized":"2024-04-23T01:47:03.285808+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.802955+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:46:14.802955+0000","last_peered":"2024-04-23T01:46:14.802955+0000","last_clean":"2024-04-23T01:46:14.802955+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:46:14.802955+0000","last_undegraded":"2024-04-23T01:46:14.802955+0000","last_fullsized":"2024-04-23T01:46:14.802955+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:47:03.285808+0000","last_change":"2024-04-23T01:47:03.285808+0000","last_active":"2024-04-23T01:47:03.285808+0000","last_peered":"2024-04-23T01:47:03.285808+0000","last_clean":"2024-04-23T01:47:03.285808+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:47:03.285808+0000","last_undegraded":"2024-04-23T01:47:03.285808+0000","last_fullsized":"2024-04-23T01:47:03.285808+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":43,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.793283+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:46:14.793283+0000","last_peered":"2024-04-23T01:46:14.793283+0000","last_clean":"2024-04-23T01:46:14.793283+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:46:14.793283+0000","last_undegraded":"2024-04-23T01:46:14.793283+0000","last_fullsized":"2024-04-23T01:46:14.793283+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:47:03.285808+0000","last_change":"2024-04-23T01:47:03.285808+0000","last_active":"2024-04-23T01:47:03.285808+0000","last_peered":"2024-04-23T01:47:03.285808+0000","last_clean":"2024-04-23T01:47:03.285808+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:47:03.285808+0000","last_undegraded":"2024-04-23T01:47:03.285808+0000","last_fullsized":"2024-04-23T01:47:03.285808+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-23T01:46:14.803589+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:46:14.803589+0000","last_peered":"2024-04-23T01:46:14.803589+0000","last_clean":"2024-04-23T01:46:14.803589+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:46:14.803589+0000","last_undegraded":"2024-04-23T01:46:14.803589+0000","last_fullsized":"2024-04-23T01:46:14.803589+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705778,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31248,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":30976,"kb_avail":94340592,"statfs":{"total":96636764160,"available":96604766208,"internally_reserved":0,"allocated":270336,"data_stored":657549,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31719424},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705778,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35408,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336432,"statfs":{"total":96636764160,"available":96600506368,"internally_reserved":0,"allocated":270336,"data_stored":657549,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705778,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":39508,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":39296,"kb_avail":94332332,"statfs":{"total":96636764160,"available":96596307968,"internally_reserved":0,"allocated":208896,"data_stored":67181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40239104},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705778,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35408,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336432,"statfs":{"total":96636764160,"available":96600506368,"internally_reserved":0,"allocated":270336,"data_stored":657549,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:47:06.380 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 1 16 152 136 1.06244 1.0625 0.0954455 0.110191 2024-04-23T01:47:07.078 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:47:07.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:07.214+0000 7f7e8382d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:07.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:07.223+0000 7f7e8382d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:07.380 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 2 16 324 308 1.20305 1.34375 0.134705 0.100761 2024-04-23T01:47:08.380 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 3 16 497 481 1.25251 1.35156 0.14121 0.0969592 2024-04-23T01:47:09.380 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 4 15 654 639 1.24797 1.23438 0.0988713 0.0986314 2024-04-23T01:47:09.403 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:47:09.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:09.548+0000 7fc7b78ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:09.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:09.557+0000 7fc7b78ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:10.380 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 5 16 810 794 1.24054 1.21094 0.107753 0.0997172 2024-04-23T01:47:11.054 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:47:11.054 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:47:11.054 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:47:11.054 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:47:11.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:11.204+0000 7f5c66f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:11.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:11.214+0000 7f5c66f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:11.245 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:47:11.246 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:47:11.246 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:47:11.246 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:47:11.246 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:47:11.246 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:47:11.246 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:47:11.246 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:47:11.246 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:47:11.246 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:47:11.246 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:47:11.246 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:47:11.250 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:47:11.250 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:47:11.251 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:47:11.381 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 6 16 977 961 1.25121 1.30469 0.077133 0.0989071 2024-04-23T01:47:11.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:11.377+0000 7fa18ee6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:11.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:11.387+0000 7fa18ee6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:11.463 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:47:11.463 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:47:11.463 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:47:11.463 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:47:11.463 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:47:11.464 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:47:11.464 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:47:11.464 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:47:11.464 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:47:11.464 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:47:11.464 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:47:11.464 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:47:11.466 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:47:11.467 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:47:11.467 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:47:11.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:11.584+0000 7fadd8f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:11.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:11.598+0000 7fadd8f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:11.619 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:47:11.619 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:47:11.619 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:47:11.619 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:47:11.619 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:47:11.620 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:47:11.620 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:47:11.620 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:47:11.620 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:47:11.620 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:47:11.620 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:47:11.620 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:47:11.622 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:47:11.623 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:47:11.623 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:47:11.762 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:47:11.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:11.765+0000 7f467962d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:11.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:11.780+0000 7f467962d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:11.802 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:47:11.802 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:47:11.802 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:47:11.802 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:47:11.802 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:47:11.802 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:47:11.802 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:47:11.802 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:47:11.802 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:47:11.803 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:47:11.803 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:47:11.803 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:47:11.805 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:47:11.806 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:47:11.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:11.918+0000 7f2894b9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:11.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:11.924+0000 7f2894b9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:12.381 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 7 16 1138 1122 1.25214 1.25781 0.133178 0.0989953 2024-04-23T01:47:13.381 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 8 16 1301 1285 1.25479 1.27344 0.117282 0.0985986 2024-04-23T01:47:14.139 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:47:14.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:14.265+0000 7f08ae0ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:14.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:14.272+0000 7f08ae0ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:14.381 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 9 16 1450 1434 1.24469 1.16406 0.122463 0.0996841 2024-04-23T01:47:15.381 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 10 16 1608 1592 1.24365 1.23438 0.0320062 0.0998913 2024-04-23T01:47:16.381 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 11 16 1765 1749 1.24209 1.22656 0.0560829 0.10011 2024-04-23T01:47:16.414 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:47:16.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:16.534+0000 7f32612ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:16.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:16.542+0000 7f32612ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:16.806 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:47:16.807 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:47:16.807 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:47:16.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:16.936+0000 7fc54515d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:16.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:16.946+0000 7fc54515d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:17.179 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:47:17.180 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:47:17.200 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2024-04-23T01:47:15.572811+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8774176,"num_objects":129,"num_object_clones":0,"num_object_copies":387,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":1136,"num_write_kb":9886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1080,"ondisk_log_size":1080,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":164456,"kb_used_data":11400,"kb_used_omap":0,"kb_used_meta":153024,"kb_avail":377322904,"statfs":{"total":386547056640,"available":386378653696,"internally_reserved":0,"allocated":11673600,"data_stored":23108288,"data_compressed":1171959,"data_compressed_allocated":10629120,"data_compressed_original":22732800,"omap_allocated":0,"internal_metadata":156696576},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8183808,"num_objects":127,"num_object_clones":0,"num_object_copies":381,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":999,"num_write_kb":7992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.260719"},"pg_stats":[{"pgid":"4.f","version":"29'79","reported_seq":92,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:11.418966+0000","last_change":"2024-04-23T01:47:04.491534+0000","last_active":"2024-04-23T01:47:11.418966+0000","last_peered":"2024-04-23T01:47:11.418966+0000","last_clean":"2024-04-23T01:47:11.418966+0000","last_became_active":"2024-04-23T01:47:04.486910+0000","last_became_peered":"2024-04-23T01:47:04.486910+0000","last_unstale":"2024-04-23T01:47:11.418966+0000","last_undegraded":"2024-04-23T01:47:11.418966+0000","last_fullsized":"2024-04-23T01:47:11.418966+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"29'39","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:11.479670+0000","last_change":"2024-04-23T01:47:04.452063+0000","last_active":"2024-04-23T01:47:11.479670+0000","last_peered":"2024-04-23T01:47:11.479670+0000","last_clean":"2024-04-23T01:47:11.479670+0000","last_became_active":"2024-04-23T01:47:04.443666+0000","last_became_peered":"2024-04-23T01:47:04.443666+0000","last_unstale":"2024-04-23T01:47:11.479670+0000","last_undegraded":"2024-04-23T01:47:11.479670+0000","last_fullsized":"2024-04-23T01:47:11.479670+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"29'79","reported_seq":92,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:11.311115+0000","last_change":"2024-04-23T01:47:04.485140+0000","last_active":"2024-04-23T01:47:11.311115+0000","last_peered":"2024-04-23T01:47:11.311115+0000","last_clean":"2024-04-23T01:47:11.311115+0000","last_became_active":"2024-04-23T01:47:04.480664+0000","last_became_peered":"2024-04-23T01:47:04.480664+0000","last_unstale":"2024-04-23T01:47:11.311115+0000","last_undegraded":"2024-04-23T01:47:11.311115+0000","last_fullsized":"2024-04-23T01:47:11.311115+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"29'63","reported_seq":76,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:09.933841+0000","last_change":"2024-04-23T01:47:04.442089+0000","last_active":"2024-04-23T01:47:09.933841+0000","last_peered":"2024-04-23T01:47:09.933841+0000","last_clean":"2024-04-23T01:47:09.933841+0000","last_became_active":"2024-04-23T01:47:04.431317+0000","last_became_peered":"2024-04-23T01:47:04.431317+0000","last_unstale":"2024-04-23T01:47:09.933841+0000","last_undegraded":"2024-04-23T01:47:09.933841+0000","last_fullsized":"2024-04-23T01:47:09.933841+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"29'55","reported_seq":68,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:10.671764+0000","last_change":"2024-04-23T01:47:04.503686+0000","last_active":"2024-04-23T01:47:10.671764+0000","last_peered":"2024-04-23T01:47:10.671764+0000","last_clean":"2024-04-23T01:47:10.671764+0000","last_became_active":"2024-04-23T01:47:04.499155+0000","last_became_peered":"2024-04-23T01:47:04.499155+0000","last_unstale":"2024-04-23T01:47:10.671764+0000","last_undegraded":"2024-04-23T01:47:10.671764+0000","last_fullsized":"2024-04-23T01:47:10.671764+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458752,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"29'63","reported_seq":76,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:11.587377+0000","last_change":"2024-04-23T01:47:04.516672+0000","last_active":"2024-04-23T01:47:11.587377+0000","last_peered":"2024-04-23T01:47:11.587377+0000","last_clean":"2024-04-23T01:47:11.587377+0000","last_became_active":"2024-04-23T01:47:04.504711+0000","last_became_peered":"2024-04-23T01:47:04.504711+0000","last_unstale":"2024-04-23T01:47:11.587377+0000","last_undegraded":"2024-04-23T01:47:11.587377+0000","last_fullsized":"2024-04-23T01:47:11.587377+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"29'47","reported_seq":60,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:11.013450+0000","last_change":"2024-04-23T01:47:04.449451+0000","last_active":"2024-04-23T01:47:11.013450+0000","last_peered":"2024-04-23T01:47:11.013450+0000","last_clean":"2024-04-23T01:47:11.013450+0000","last_became_active":"2024-04-23T01:47:04.442623+0000","last_became_peered":"2024-04-23T01:47:04.442623+0000","last_unstale":"2024-04-23T01:47:11.013450+0000","last_undegraded":"2024-04-23T01:47:11.013450+0000","last_fullsized":"2024-04-23T01:47:11.013450+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"29'31","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:10.268544+0000","last_change":"2024-04-23T01:47:04.520179+0000","last_active":"2024-04-23T01:47:10.268544+0000","last_peered":"2024-04-23T01:47:10.268544+0000","last_clean":"2024-04-23T01:47:10.268544+0000","last_became_active":"2024-04-23T01:47:04.511239+0000","last_became_peered":"2024-04-23T01:47:04.511239+0000","last_unstale":"2024-04-23T01:47:10.268544+0000","last_undegraded":"2024-04-23T01:47:10.268544+0000","last_fullsized":"2024-04-23T01:47:10.268544+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"29'71","reported_seq":84,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:10.788404+0000","last_change":"2024-04-23T01:47:04.496716+0000","last_active":"2024-04-23T01:47:10.788404+0000","last_peered":"2024-04-23T01:47:10.788404+0000","last_clean":"2024-04-23T01:47:10.788404+0000","last_became_active":"2024-04-23T01:47:04.486607+0000","last_became_peered":"2024-04-23T01:47:04.486607+0000","last_unstale":"2024-04-23T01:47:10.788404+0000","last_undegraded":"2024-04-23T01:47:10.788404+0000","last_fullsized":"2024-04-23T01:47:10.788404+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":186,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.325282+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:47:05.325282+0000","last_peered":"2024-04-23T01:47:05.325282+0000","last_clean":"2024-04-23T01:47:05.325282+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:47:05.325282+0000","last_undegraded":"2024-04-23T01:47:05.325282+0000","last_fullsized":"2024-04-23T01:47:05.325282+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.410555+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:47:05.410555+0000","last_peered":"2024-04-23T01:47:05.410555+0000","last_clean":"2024-04-23T01:47:05.410555+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:47:05.410555+0000","last_undegraded":"2024-04-23T01:47:05.410555+0000","last_fullsized":"2024-04-23T01:47:05.410555+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"29'48","reported_seq":61,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:11.146586+0000","last_change":"2024-04-23T01:47:04.467386+0000","last_active":"2024-04-23T01:47:11.146586+0000","last_peered":"2024-04-23T01:47:11.146586+0000","last_clean":"2024-04-23T01:47:11.146586+0000","last_became_active":"2024-04-23T01:47:04.459042+0000","last_became_peered":"2024-04-23T01:47:04.459042+0000","last_unstale":"2024-04-23T01:47:11.146586+0000","last_undegraded":"2024-04-23T01:47:11.146586+0000","last_fullsized":"2024-04-23T01:47:11.146586+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":401408,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.570797+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:47:05.570797+0000","last_peered":"2024-04-23T01:47:05.570797+0000","last_clean":"2024-04-23T01:47:05.570797+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:47:05.570797+0000","last_undegraded":"2024-04-23T01:47:05.570797+0000","last_fullsized":"2024-04-23T01:47:05.570797+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"29'80","reported_seq":93,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:11.076364+0000","last_change":"2024-04-23T01:47:04.466464+0000","last_active":"2024-04-23T01:47:11.076364+0000","last_peered":"2024-04-23T01:47:11.076364+0000","last_clean":"2024-04-23T01:47:11.076364+0000","last_became_active":"2024-04-23T01:47:04.460338+0000","last_became_peered":"2024-04-23T01:47:04.460338+0000","last_unstale":"2024-04-23T01:47:11.076364+0000","last_undegraded":"2024-04-23T01:47:11.076364+0000","last_fullsized":"2024-04-23T01:47:11.076364+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":663552,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81,"num_write_kb":648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.397538+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:47:05.397538+0000","last_peered":"2024-04-23T01:47:05.397538+0000","last_clean":"2024-04-23T01:47:05.397538+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:47:05.397538+0000","last_undegraded":"2024-04-23T01:47:05.397538+0000","last_fullsized":"2024-04-23T01:47:05.397538+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"29'87","reported_seq":100,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:11.371891+0000","last_change":"2024-04-23T01:47:04.435560+0000","last_active":"2024-04-23T01:47:11.371891+0000","last_peered":"2024-04-23T01:47:11.371891+0000","last_clean":"2024-04-23T01:47:11.371891+0000","last_became_active":"2024-04-23T01:47:04.424493+0000","last_became_peered":"2024-04-23T01:47:04.424493+0000","last_unstale":"2024-04-23T01:47:11.371891+0000","last_undegraded":"2024-04-23T01:47:11.371891+0000","last_fullsized":"2024-04-23T01:47:11.371891+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":720896,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.576933+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:47:05.576933+0000","last_peered":"2024-04-23T01:47:05.576933+0000","last_clean":"2024-04-23T01:47:05.576933+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:47:05.576933+0000","last_undegraded":"2024-04-23T01:47:05.576933+0000","last_fullsized":"2024-04-23T01:47:05.576933+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"29'47","reported_seq":60,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:11.503041+0000","last_change":"2024-04-23T01:47:04.530152+0000","last_active":"2024-04-23T01:47:11.503041+0000","last_peered":"2024-04-23T01:47:11.503041+0000","last_clean":"2024-04-23T01:47:11.503041+0000","last_became_active":"2024-04-23T01:47:04.519844+0000","last_became_peered":"2024-04-23T01:47:04.519844+0000","last_unstale":"2024-04-23T01:47:11.503041+0000","last_undegraded":"2024-04-23T01:47:11.503041+0000","last_fullsized":"2024-04-23T01:47:11.503041+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.590971+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:47:05.590971+0000","last_peered":"2024-04-23T01:47:05.590971+0000","last_clean":"2024-04-23T01:47:05.590971+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:47:05.590971+0000","last_undegraded":"2024-04-23T01:47:05.590971+0000","last_fullsized":"2024-04-23T01:47:05.590971+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"29'71","reported_seq":84,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:10.930157+0000","last_change":"2024-04-23T01:47:04.465832+0000","last_active":"2024-04-23T01:47:10.930157+0000","last_peered":"2024-04-23T01:47:10.930157+0000","last_clean":"2024-04-23T01:47:10.930157+0000","last_became_active":"2024-04-23T01:47:04.460695+0000","last_became_peered":"2024-04-23T01:47:04.460695+0000","last_unstale":"2024-04-23T01:47:10.930157+0000","last_undegraded":"2024-04-23T01:47:10.930157+0000","last_fullsized":"2024-04-23T01:47:10.930157+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.331332+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:47:05.331332+0000","last_peered":"2024-04-23T01:47:05.331332+0000","last_clean":"2024-04-23T01:47:05.331332+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:47:05.331332+0000","last_undegraded":"2024-04-23T01:47:05.331332+0000","last_fullsized":"2024-04-23T01:47:05.331332+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"29'68","reported_seq":81,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:11.559296+0000","last_change":"2024-04-23T01:47:04.434787+0000","last_active":"2024-04-23T01:47:11.559296+0000","last_peered":"2024-04-23T01:47:11.559296+0000","last_clean":"2024-04-23T01:47:11.559296+0000","last_became_active":"2024-04-23T01:47:04.423352+0000","last_became_peered":"2024-04-23T01:47:04.423352+0000","last_unstale":"2024-04-23T01:47:11.559296+0000","last_undegraded":"2024-04-23T01:47:11.559296+0000","last_fullsized":"2024-04-23T01:47:11.559296+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":565248,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69,"num_write_kb":552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.602643+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:47:05.602643+0000","last_peered":"2024-04-23T01:47:05.602643+0000","last_clean":"2024-04-23T01:47:05.602643+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:47:05.602643+0000","last_undegraded":"2024-04-23T01:47:05.602643+0000","last_fullsized":"2024-04-23T01:47:05.602643+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"29'55","reported_seq":68,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:11.248388+0000","last_change":"2024-04-23T01:47:04.425726+0000","last_active":"2024-04-23T01:47:11.248388+0000","last_peered":"2024-04-23T01:47:11.248388+0000","last_clean":"2024-04-23T01:47:11.248388+0000","last_became_active":"2024-04-23T01:47:04.416905+0000","last_became_peered":"2024-04-23T01:47:04.416905+0000","last_unstale":"2024-04-23T01:47:11.248388+0000","last_undegraded":"2024-04-23T01:47:11.248388+0000","last_fullsized":"2024-04-23T01:47:11.248388+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458752,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.400601+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:47:05.400601+0000","last_peered":"2024-04-23T01:47:05.400601+0000","last_clean":"2024-04-23T01:47:05.400601+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:47:05.400601+0000","last_undegraded":"2024-04-23T01:47:05.400601+0000","last_fullsized":"2024-04-23T01:47:05.400601+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":8183808,"num_objects":127,"num_object_clones":0,"num_object_copies":381,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":999,"num_write_kb":7992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12267520,"data_stored":24535040,"data_compressed":1263567,"data_compressed_allocated":12267520,"data_compressed_original":24535040,"omap_allocated":0,"internal_metadata":0},"log_size":983,"ondisk_log_size":983,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705788,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":34004,"kb_used_data":3020,"kb_used_omap":0,"kb_used_meta":30976,"kb_avail":94337836,"statfs":{"total":96636764160,"available":96601944064,"internally_reserved":0,"allocated":3092480,"data_stored":6242104,"data_compressed":316843,"data_compressed_allocated":2826240,"data_compressed_original":6144000,"omap_allocated":0,"internal_metadata":31719424},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705788,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41772,"kb_used_data":2468,"kb_used_omap":0,"kb_used_meta":39296,"kb_avail":94330068,"statfs":{"total":96636764160,"available":96593989632,"internally_reserved":0,"allocated":2527232,"data_stored":5111608,"data_compressed":258605,"data_compressed_allocated":2260992,"data_compressed_original":5013504,"omap_allocated":0,"internal_metadata":40239104},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705788,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46364,"kb_used_data":2900,"kb_used_omap":0,"kb_used_meta":43456,"kb_avail":94325476,"statfs":{"total":96636764160,"available":96589287424,"internally_reserved":0,"allocated":2969600,"data_stored":5528856,"data_compressed":280528,"data_compressed_allocated":2723840,"data_compressed_original":5447680,"omap_allocated":0,"internal_metadata":44498944},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705788,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":42316,"kb_used_data":3012,"kb_used_omap":0,"kb_used_meta":39296,"kb_avail":94329524,"statfs":{"total":96636764160,"available":96593432576,"internally_reserved":0,"allocated":3084288,"data_stored":6225720,"data_compressed":315983,"data_compressed_allocated":2818048,"data_compressed_original":6127616,"omap_allocated":0,"internal_metadata":40239104},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3112960,"data_stored":6225920,"data_compressed":320637,"data_compressed_allocated":3112960,"data_compressed_original":6225920,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3231744,"data_stored":6463488,"data_compressed":332856,"data_compressed_allocated":3231744,"data_compressed_original":6463488,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2686976,"data_stored":5373952,"data_compressed":276762,"data_compressed_allocated":2686976,"data_compressed_original":5373952,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3235840,"data_stored":6471680,"data_compressed":333312,"data_compressed_allocated":3235840,"data_compressed_original":6471680,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:47:17.381 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 12 16 1916 1900 1.23688 1.17969 0.147344 0.100344 2024-04-23T01:47:18.381 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 13 16 2081 2065 1.24089 1.28906 0.130745 0.100392 2024-04-23T01:47:18.732 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:47:18.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:18.877+0000 7f31ab39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:18.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:18.886+0000 7f31ab39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:19.381 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 14 16 2243 2227 1.24265 1.26562 0.0817145 0.100143 2024-04-23T01:47:20.381 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 15 16 2415 2399 1.24938 1.34375 0.0457613 0.099572 2024-04-23T01:47:21.052 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:47:21.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:21.169+0000 7f4d3182d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:21.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:21.178+0000 7f4d3182d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:21.381 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 16 16 2565 2549 1.24453 1.17188 0.0282841 0.0998894 2024-04-23T01:47:22.203 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:47:22.203 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:47:22.203 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:47:22.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:22.326+0000 7f2ebea2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:22.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:22.335+0000 7f2ebea2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:22.381 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 17 16 2722 2706 1.24347 1.22656 0.0764937 0.100027 2024-04-23T01:47:22.564 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:47:22.564 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:47:22.583 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2024-04-23T01:47:21.573840+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":15221280,"num_objects":227,"num_object_clones":0,"num_object_copies":681,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":1923,"num_write_kb":16182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1867,"ondisk_log_size":1867,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":161452,"kb_used_data":20876,"kb_used_omap":0,"kb_used_meta":140544,"kb_avail":377325908,"statfs":{"total":386547056640,"available":386381729792,"internally_reserved":0,"allocated":21377024,"data_stored":42515136,"data_compressed":2173611,"data_compressed_allocated":20332544,"data_compressed_original":42139648,"omap_allocated":0,"internal_metadata":143917056},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13148160,"num_objects":201,"num_object_clones":0,"num_object_copies":603,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1605,"num_write_kb":12840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002148"},"pg_stats":[{"pgid":"4.f","version":"29'139","reported_seq":152,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:16.562949+0000","last_change":"2024-04-23T01:47:04.491534+0000","last_active":"2024-04-23T01:47:16.562949+0000","last_peered":"2024-04-23T01:47:16.562949+0000","last_clean":"2024-04-23T01:47:16.562949+0000","last_became_active":"2024-04-23T01:47:04.486910+0000","last_became_peered":"2024-04-23T01:47:04.486910+0000","last_unstale":"2024-04-23T01:47:16.562949+0000","last_undegraded":"2024-04-23T01:47:16.562949+0000","last_fullsized":"2024-04-23T01:47:16.562949+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1146880,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":1120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"29'71","reported_seq":84,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:14.461728+0000","last_change":"2024-04-23T01:47:04.452063+0000","last_active":"2024-04-23T01:47:14.461728+0000","last_peered":"2024-04-23T01:47:14.461728+0000","last_clean":"2024-04-23T01:47:14.461728+0000","last_became_active":"2024-04-23T01:47:04.443666+0000","last_became_peered":"2024-04-23T01:47:04.443666+0000","last_unstale":"2024-04-23T01:47:14.461728+0000","last_undegraded":"2024-04-23T01:47:14.461728+0000","last_fullsized":"2024-04-23T01:47:14.461728+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"29'99","reported_seq":112,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:16.559265+0000","last_change":"2024-04-23T01:47:04.485140+0000","last_active":"2024-04-23T01:47:16.559265+0000","last_peered":"2024-04-23T01:47:16.559265+0000","last_clean":"2024-04-23T01:47:16.559265+0000","last_became_active":"2024-04-23T01:47:04.480664+0000","last_became_peered":"2024-04-23T01:47:04.480664+0000","last_unstale":"2024-04-23T01:47:16.559265+0000","last_undegraded":"2024-04-23T01:47:16.559265+0000","last_fullsized":"2024-04-23T01:47:16.559265+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":819200,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"29'79","reported_seq":92,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:16.450183+0000","last_change":"2024-04-23T01:47:04.442089+0000","last_active":"2024-04-23T01:47:16.450183+0000","last_peered":"2024-04-23T01:47:16.450183+0000","last_clean":"2024-04-23T01:47:16.450183+0000","last_became_active":"2024-04-23T01:47:04.431317+0000","last_became_peered":"2024-04-23T01:47:04.431317+0000","last_unstale":"2024-04-23T01:47:16.450183+0000","last_undegraded":"2024-04-23T01:47:16.450183+0000","last_fullsized":"2024-04-23T01:47:16.450183+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"29'103","reported_seq":116,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:16.403455+0000","last_change":"2024-04-23T01:47:04.503686+0000","last_active":"2024-04-23T01:47:16.403455+0000","last_peered":"2024-04-23T01:47:16.403455+0000","last_clean":"2024-04-23T01:47:16.403455+0000","last_became_active":"2024-04-23T01:47:04.499155+0000","last_became_peered":"2024-04-23T01:47:04.499155+0000","last_unstale":"2024-04-23T01:47:16.403455+0000","last_undegraded":"2024-04-23T01:47:16.403455+0000","last_fullsized":"2024-04-23T01:47:16.403455+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"29'159","reported_seq":172,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:15.968807+0000","last_change":"2024-04-23T01:47:04.516672+0000","last_active":"2024-04-23T01:47:15.968807+0000","last_peered":"2024-04-23T01:47:15.968807+0000","last_clean":"2024-04-23T01:47:15.968807+0000","last_became_active":"2024-04-23T01:47:04.504711+0000","last_became_peered":"2024-04-23T01:47:04.504711+0000","last_unstale":"2024-04-23T01:47:15.968807+0000","last_undegraded":"2024-04-23T01:47:15.968807+0000","last_fullsized":"2024-04-23T01:47:15.968807+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310720,"num_objects":20,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":160,"num_write_kb":1280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"29'103","reported_seq":116,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:16.302473+0000","last_change":"2024-04-23T01:47:04.449451+0000","last_active":"2024-04-23T01:47:16.302473+0000","last_peered":"2024-04-23T01:47:16.302473+0000","last_clean":"2024-04-23T01:47:16.302473+0000","last_became_active":"2024-04-23T01:47:04.442623+0000","last_became_peered":"2024-04-23T01:47:04.442623+0000","last_unstale":"2024-04-23T01:47:16.302473+0000","last_undegraded":"2024-04-23T01:47:16.302473+0000","last_fullsized":"2024-04-23T01:47:16.302473+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"29'87","reported_seq":100,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:15.177901+0000","last_change":"2024-04-23T01:47:04.520179+0000","last_active":"2024-04-23T01:47:15.177901+0000","last_peered":"2024-04-23T01:47:15.177901+0000","last_clean":"2024-04-23T01:47:15.177901+0000","last_became_active":"2024-04-23T01:47:04.511239+0000","last_became_peered":"2024-04-23T01:47:04.511239+0000","last_unstale":"2024-04-23T01:47:15.177901+0000","last_undegraded":"2024-04-23T01:47:15.177901+0000","last_fullsized":"2024-04-23T01:47:15.177901+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":720896,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"29'103","reported_seq":116,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:15.767044+0000","last_change":"2024-04-23T01:47:04.496716+0000","last_active":"2024-04-23T01:47:15.767044+0000","last_peered":"2024-04-23T01:47:15.767044+0000","last_clean":"2024-04-23T01:47:15.767044+0000","last_became_active":"2024-04-23T01:47:04.486607+0000","last_became_peered":"2024-04-23T01:47:04.486607+0000","last_unstale":"2024-04-23T01:47:15.767044+0000","last_undegraded":"2024-04-23T01:47:15.767044+0000","last_fullsized":"2024-04-23T01:47:15.767044+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":186,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.325282+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:47:05.325282+0000","last_peered":"2024-04-23T01:47:05.325282+0000","last_clean":"2024-04-23T01:47:05.325282+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:47:05.325282+0000","last_undegraded":"2024-04-23T01:47:05.325282+0000","last_fullsized":"2024-04-23T01:47:05.325282+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.410555+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:47:05.410555+0000","last_peered":"2024-04-23T01:47:05.410555+0000","last_clean":"2024-04-23T01:47:05.410555+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:47:05.410555+0000","last_undegraded":"2024-04-23T01:47:05.410555+0000","last_fullsized":"2024-04-23T01:47:05.410555+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"29'89","reported_seq":102,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:16.566952+0000","last_change":"2024-04-23T01:47:04.467386+0000","last_active":"2024-04-23T01:47:16.566952+0000","last_peered":"2024-04-23T01:47:16.566952+0000","last_clean":"2024-04-23T01:47:16.566952+0000","last_became_active":"2024-04-23T01:47:04.459042+0000","last_became_peered":"2024-04-23T01:47:04.459042+0000","last_unstale":"2024-04-23T01:47:16.566952+0000","last_undegraded":"2024-04-23T01:47:16.566952+0000","last_fullsized":"2024-04-23T01:47:16.566952+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":737280,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90,"num_write_kb":720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.570797+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:47:05.570797+0000","last_peered":"2024-04-23T01:47:05.570797+0000","last_clean":"2024-04-23T01:47:05.570797+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:47:05.570797+0000","last_undegraded":"2024-04-23T01:47:05.570797+0000","last_fullsized":"2024-04-23T01:47:05.570797+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"29'119","reported_seq":132,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:15.442360+0000","last_change":"2024-04-23T01:47:04.466464+0000","last_active":"2024-04-23T01:47:15.442360+0000","last_peered":"2024-04-23T01:47:15.442360+0000","last_clean":"2024-04-23T01:47:15.442360+0000","last_became_active":"2024-04-23T01:47:04.460338+0000","last_became_peered":"2024-04-23T01:47:04.460338+0000","last_unstale":"2024-04-23T01:47:15.442360+0000","last_undegraded":"2024-04-23T01:47:15.442360+0000","last_fullsized":"2024-04-23T01:47:15.442360+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.397538+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:47:05.397538+0000","last_peered":"2024-04-23T01:47:05.397538+0000","last_clean":"2024-04-23T01:47:05.397538+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:47:05.397538+0000","last_undegraded":"2024-04-23T01:47:05.397538+0000","last_fullsized":"2024-04-23T01:47:05.397538+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"29'119","reported_seq":132,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:16.522790+0000","last_change":"2024-04-23T01:47:04.435560+0000","last_active":"2024-04-23T01:47:16.522790+0000","last_peered":"2024-04-23T01:47:16.522790+0000","last_clean":"2024-04-23T01:47:16.522790+0000","last_became_active":"2024-04-23T01:47:04.424493+0000","last_became_peered":"2024-04-23T01:47:04.424493+0000","last_unstale":"2024-04-23T01:47:16.522790+0000","last_undegraded":"2024-04-23T01:47:16.522790+0000","last_fullsized":"2024-04-23T01:47:16.522790+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.576933+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:47:05.576933+0000","last_peered":"2024-04-23T01:47:05.576933+0000","last_clean":"2024-04-23T01:47:05.576933+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:47:05.576933+0000","last_undegraded":"2024-04-23T01:47:05.576933+0000","last_fullsized":"2024-04-23T01:47:05.576933+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"29'119","reported_seq":132,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:16.089540+0000","last_change":"2024-04-23T01:47:04.530152+0000","last_active":"2024-04-23T01:47:16.089540+0000","last_peered":"2024-04-23T01:47:16.089540+0000","last_clean":"2024-04-23T01:47:16.089540+0000","last_became_active":"2024-04-23T01:47:04.519844+0000","last_became_peered":"2024-04-23T01:47:04.519844+0000","last_unstale":"2024-04-23T01:47:16.089540+0000","last_undegraded":"2024-04-23T01:47:16.089540+0000","last_fullsized":"2024-04-23T01:47:16.089540+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.590971+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:47:05.590971+0000","last_peered":"2024-04-23T01:47:05.590971+0000","last_clean":"2024-04-23T01:47:05.590971+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:47:05.590971+0000","last_undegraded":"2024-04-23T01:47:05.590971+0000","last_fullsized":"2024-04-23T01:47:05.590971+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"29'143","reported_seq":156,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:16.212041+0000","last_change":"2024-04-23T01:47:04.465832+0000","last_active":"2024-04-23T01:47:16.212041+0000","last_peered":"2024-04-23T01:47:16.212041+0000","last_clean":"2024-04-23T01:47:16.212041+0000","last_became_active":"2024-04-23T01:47:04.460695+0000","last_became_peered":"2024-04-23T01:47:04.460695+0000","last_unstale":"2024-04-23T01:47:16.212041+0000","last_undegraded":"2024-04-23T01:47:16.212041+0000","last_fullsized":"2024-04-23T01:47:16.212041+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.331332+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:47:05.331332+0000","last_peered":"2024-04-23T01:47:05.331332+0000","last_clean":"2024-04-23T01:47:05.331332+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:47:05.331332+0000","last_undegraded":"2024-04-23T01:47:05.331332+0000","last_fullsized":"2024-04-23T01:47:05.331332+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"29'135","reported_seq":148,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:14.982318+0000","last_change":"2024-04-23T01:47:04.434787+0000","last_active":"2024-04-23T01:47:14.982318+0000","last_peered":"2024-04-23T01:47:14.982318+0000","last_clean":"2024-04-23T01:47:14.982318+0000","last_became_active":"2024-04-23T01:47:04.423352+0000","last_became_peered":"2024-04-23T01:47:04.423352+0000","last_unstale":"2024-04-23T01:47:14.982318+0000","last_undegraded":"2024-04-23T01:47:14.982318+0000","last_fullsized":"2024-04-23T01:47:14.982318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":136,"num_write_kb":1088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.602643+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:47:05.602643+0000","last_peered":"2024-04-23T01:47:05.602643+0000","last_clean":"2024-04-23T01:47:05.602643+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:47:05.602643+0000","last_undegraded":"2024-04-23T01:47:05.602643+0000","last_fullsized":"2024-04-23T01:47:05.602643+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"29'103","reported_seq":116,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:16.562707+0000","last_change":"2024-04-23T01:47:04.425726+0000","last_active":"2024-04-23T01:47:16.562707+0000","last_peered":"2024-04-23T01:47:16.562707+0000","last_clean":"2024-04-23T01:47:16.562707+0000","last_became_active":"2024-04-23T01:47:04.416905+0000","last_became_peered":"2024-04-23T01:47:04.416905+0000","last_unstale":"2024-04-23T01:47:16.562707+0000","last_undegraded":"2024-04-23T01:47:16.562707+0000","last_fullsized":"2024-04-23T01:47:16.562707+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.400601+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:47:05.400601+0000","last_peered":"2024-04-23T01:47:05.400601+0000","last_clean":"2024-04-23T01:47:05.400601+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:47:05.400601+0000","last_undegraded":"2024-04-23T01:47:05.400601+0000","last_fullsized":"2024-04-23T01:47:05.400601+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":14630912,"num_objects":225,"num_object_clones":0,"num_object_copies":675,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1786,"num_write_kb":14288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":21929984,"data_stored":43859968,"data_compressed":2261412,"data_compressed_allocated":21929984,"data_compressed_original":43859968,"omap_allocated":0,"internal_metadata":0},"log_size":1770,"ondisk_log_size":1770,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705793,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40744,"kb_used_data":5600,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94331096,"statfs":{"total":96636764160,"available":96595042304,"internally_reserved":0,"allocated":5734400,"data_stored":11525944,"data_compressed":589566,"data_compressed_allocated":5468160,"data_compressed_original":11427840,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705793,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":43916,"kb_used_data":4612,"kb_used_omap":0,"kb_used_meta":39296,"kb_avail":94327924,"statfs":{"total":96636764160,"available":96591794176,"internally_reserved":0,"allocated":4722688,"data_stored":9502520,"data_compressed":485219,"data_compressed_allocated":4456448,"data_compressed_original":9404416,"omap_allocated":0,"internal_metadata":40239104},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705793,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32356,"kb_used_data":5532,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94339484,"statfs":{"total":96636764160,"available":96603631616,"internally_reserved":0,"allocated":5664768,"data_stored":10919192,"data_compressed":558735,"data_compressed_allocated":5419008,"data_compressed_original":10838016,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705793,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":44436,"kb_used_data":5132,"kb_used_omap":0,"kb_used_meta":39296,"kb_avail":94327404,"statfs":{"total":96636764160,"available":96591261696,"internally_reserved":0,"allocated":5255168,"data_stored":10567480,"data_compressed":540091,"data_compressed_allocated":4988928,"data_compressed_original":10469376,"omap_allocated":0,"internal_metadata":40239104},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5382144,"data_stored":10764288,"data_compressed":554976,"data_compressed_allocated":5382144,"data_compressed_original":10764288,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5881856,"data_stored":11763712,"data_compressed":606534,"data_compressed_allocated":5881856,"data_compressed_original":11763712,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4784128,"data_stored":9568256,"data_compressed":493336,"data_compressed_allocated":4784128,"data_compressed_original":9568256,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5881856,"data_stored":11763712,"data_compressed":606566,"data_compressed_allocated":5881856,"data_compressed_original":11763712,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:47:23.313 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:47:23.382 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 18 16 2867 2851 1.23731 1.13281 0.242929 0.100659 2024-04-23T01:47:23.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:23.455+0000 7f0e82a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:23.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:23.463+0000 7f0e82a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:24.382 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 19 16 3025 3009 1.23715 1.23438 0.0954475 0.100728 2024-04-23T01:47:25.382 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:47:25.383812+0000 min lat: 0.0142109 max lat: 0.30834 avg lat: 0.100964 2024-04-23T01:47:25.382 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:47:25.382 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 20 16 3176 3160 1.23427 1.17969 0.116172 0.100964 2024-04-23T01:47:25.641 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:47:25.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:25.773+0000 7f62d555d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:25.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:25.780+0000 7f62d555d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:26.382 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 21 16 3340 3324 1.2365 1.28125 0.152309 0.100863 2024-04-23T01:47:27.382 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 22 16 3506 3490 1.23924 1.29688 0.0657305 0.100664 2024-04-23T01:47:27.586 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:47:27.586 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:47:27.586 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:47:27.586 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:47:27.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:27.732+0000 7fa79635d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:27.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:27.745+0000 7fa79635d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:27.775 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:47:27.776 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:47:27.776 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:47:27.776 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:47:27.776 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:47:27.776 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:47:27.776 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:47:27.776 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:47:27.776 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:47:27.776 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:47:27.776 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:47:27.777 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:47:27.779 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:47:27.780 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:47:27.780 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:47:27.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:27.910+0000 7f68becad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:27.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:27.920+0000 7f68becad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:27.982 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:47:28.038 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:47:28.039 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:47:28.039 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:47:28.039 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:47:28.039 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:47:28.039 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:47:28.039 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:47:28.039 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:47:28.039 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:47:28.039 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:47:28.039 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:47:28.039 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:47:28.043 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:47:28.043 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:47:28.043 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:47:28.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:28.113+0000 7fd39bf5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:28.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:28.122+0000 7fd39bf5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:28.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:28.178+0000 7f1f3e19d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:28.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:28.191+0000 7f1f3e19d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:28.220 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:47:28.220 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:47:28.220 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:47:28.220 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:47:28.220 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:47:28.220 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:47:28.220 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:47:28.220 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:47:28.220 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:47:28.220 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:47:28.221 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:47:28.221 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:47:28.221 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:47:28.222 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:47:28.222 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:47:28.382 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 23 16 3664 3648 1.23903 1.23438 0.129087 0.100475 2024-04-23T01:47:28.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:28.385+0000 7f75e18ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:28.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:28.399+0000 7f75e18ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:28.511 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:47:28.511 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:47:28.511 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:47:28.511 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:47:28.511 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:47:28.511 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:47:28.511 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:47:28.511 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:47:28.511 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:47:28.511 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:47:28.511 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:47:28.512 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:47:28.514 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:47:28.515 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:47:29.382 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 24 16 3825 3809 1.23981 1.25781 0.0961603 0.100635 2024-04-23T01:47:30.347 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:47:30.382 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 25 16 3972 3956 1.23615 1.14844 0.0256896 0.100849 2024-04-23T01:47:30.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:30.479+0000 7f4fc202d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:30.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:30.485+0000 7f4fc202d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:31.382 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 26 16 4127 4111 1.23517 1.21094 0.109185 0.100951 2024-04-23T01:47:32.382 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 27 16 4290 4274 1.23659 1.27344 0.0838998 0.100807 2024-04-23T01:47:32.629 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:47:32.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:32.771+0000 7fbf1eaed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:32.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:32.777+0000 7fbf1eaed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:33.382 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 28 16 4455 4439 1.23846 1.28906 0.113048 0.100763 2024-04-23T01:47:33.516 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:47:33.516 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:47:33.516 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:47:33.516 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:47:33.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:33.636+0000 7f653959d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:33.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:33.649+0000 7f653959d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:33.676 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:47:33.676 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:47:33.676 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:47:33.676 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:47:33.676 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:47:33.677 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:47:33.677 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:47:33.677 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:47:33.677 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:47:33.677 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:47:33.677 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:47:33.677 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:47:33.680 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:47:33.680 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:47:33.681 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:47:33.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:33.819+0000 7ffa34a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:33.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:33.831+0000 7ffa34a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:33.849 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:47:33.849 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:47:33.849 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:47:33.849 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:47:33.849 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:47:33.849 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:47:33.849 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:47:33.850 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:47:33.850 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:47:33.850 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:47:33.850 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:47:33.850 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:47:33.853 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:47:33.853 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:47:33.853 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:47:33.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:33.987+0000 7f2e5179d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:33.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:33.999+0000 7f2e5179d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:34.014 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:47:34.014 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:47:34.014 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:47:34.014 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:47:34.014 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:47:34.015 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:47:34.015 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:47:34.015 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:47:34.015 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:47:34.015 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:47:34.015 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:47:34.015 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:47:34.017 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:47:34.018 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:47:34.018 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:47:34.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:34.148+0000 7f036662d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:34.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:34.159+0000 7f036662d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:34.192 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:47:34.192 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:47:34.193 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:47:34.193 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:47:34.193 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:47:34.193 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:47:34.193 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:47:34.193 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:47:34.193 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:47:34.193 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:47:34.193 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:47:34.193 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:47:34.195 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:47:34.196 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:47:34.383 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 29 16 4597 4581 1.234 1.10938 0.030264 0.101053 2024-04-23T01:47:34.999 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:47:35.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:35.155+0000 7f94db95d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:35.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:35.163+0000 7f94db95d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:35.383 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 30 16 4748 4732 1.23219 1.17969 0.0402421 0.101261 2024-04-23T01:47:36.383 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 31 15 4903 4888 1.23175 1.21875 0.166469 0.101212 2024-04-23T01:47:37.307 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:47:37.383 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 32 16 5043 5027 1.22719 1.08594 0.0809859 0.101591 2024-04-23T01:47:37.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:37.428+0000 7f71d68ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:37.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:37.433+0000 7f71d68ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:38.383 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 33 16 5190 5174 1.2248 1.14844 0.0815333 0.101891 2024-04-23T01:47:39.196 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:47:39.196 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:47:39.196 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:47:39.197 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:47:39.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:39.314+0000 7f99eacad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:39.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:39.324+0000 7f99eacad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:39.345 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:47:39.345 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:47:39.345 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:47:39.345 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:47:39.345 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:47:39.345 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:47:39.345 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:47:39.345 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:47:39.345 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:47:39.345 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:47:39.345 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:47:39.345 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:47:39.348 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:47:39.348 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:47:39.349 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:47:39.383 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 34 16 5325 5309 1.2198 1.05469 0.198631 0.102213 2024-04-23T01:47:39.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:39.483+0000 7f2b1459d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:39.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:39.495+0000 7f2b1459d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:39.521 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:47:39.521 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:47:39.521 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:47:39.521 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:47:39.521 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:47:39.522 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:47:39.522 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:47:39.522 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:47:39.522 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:47:39.522 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:47:39.522 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:47:39.522 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:47:39.524 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:47:39.524 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:47:39.525 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:47:39.651 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:47:39.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:39.696+0000 7fb9f059d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:39.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:39.710+0000 7fb9f059d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:39.729 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:47:39.729 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:47:39.729 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:47:39.729 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:47:39.729 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:47:39.729 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:47:39.729 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:47:39.729 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:47:39.730 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:47:39.730 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:47:39.730 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:47:39.730 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:47:39.732 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:47:39.732 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:47:39.733 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:47:39.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:39.808+0000 7f75426ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:39.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:39.815+0000 7f75426ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:39.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:39.873+0000 7f2158aed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:39.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:39.885+0000 7f2158aed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:39.902 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:47:39.902 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:47:39.902 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:47:39.902 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:47:39.902 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:47:39.902 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:47:39.902 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:47:39.902 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:47:39.903 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:47:39.903 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:47:39.903 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:47:39.903 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:47:39.905 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:47:39.906 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:47:40.383 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 35 16 5465 5449 1.21619 1.09375 0.0667228 0.102587 2024-04-23T01:47:41.383 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 36 16 5607 5591 1.21322 1.10938 0.140399 0.102772 2024-04-23T01:47:41.959 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:47:42.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:42.080+0000 7f940d954640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:42.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:42.089+0000 7f940d954640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:42.383 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 37 16 5751 5735 1.21084 1.125 0.0809843 0.103035 2024-04-23T01:47:43.383 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 38 16 5902 5886 1.21001 1.17969 0.0566701 0.103115 2024-04-23T01:47:44.282 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:47:44.384 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 39 16 6051 6035 1.20883 1.16406 0.021185 0.103159 2024-04-23T01:47:44.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:44.410+0000 7ff63666d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:44.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:44.423+0000 7ff63666d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:44.906 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:47:44.907 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:47:44.907 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:47:44.907 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:47:45.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:45.030+0000 7f6f22624640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:45.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:45.040+0000 7f6f22624640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:45.064 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:47:45.064 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:47:45.065 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:47:45.065 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:47:45.065 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:47:45.065 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:47:45.065 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:47:45.065 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:47:45.065 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:47:45.065 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:47:45.065 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:47:45.065 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:47:45.067 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:47:45.068 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:47:45.068 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:47:45.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:45.205+0000 7fcdf2eed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:45.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:45.215+0000 7fcdf2eed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:45.276 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:47:45.277 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:47:45.277 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:47:45.277 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:47:45.277 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:47:45.277 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:47:45.277 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:47:45.277 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:47:45.277 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:47:45.277 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:47:45.277 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:47:45.277 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:47:45.280 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:47:45.280 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:47:45.281 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:47:45.384 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:47:45.385555+0000 min lat: 0.0142109 max lat: 0.326408 avg lat: 0.103919 2024-04-23T01:47:45.384 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:47:45.384 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 40 16 6161 6145 1.20009 0.859375 0.0740836 0.103919 2024-04-23T01:47:45.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:45.421+0000 7f66b0b9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:45.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:45.434+0000 7f66b0b9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:45.456 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:47:45.457 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:47:45.457 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:47:45.457 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:47:45.457 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:47:45.457 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:47:45.457 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:47:45.457 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:47:45.457 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:47:45.457 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:47:45.457 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:47:45.458 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:47:45.459 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:47:45.460 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:47:45.460 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:47:45.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:45.592+0000 7fc72722d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:45.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:45.603+0000 7fc72722d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:45.657 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:47:45.657 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:47:45.657 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:47:45.657 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:47:45.658 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:47:45.658 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:47:45.658 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:47:45.658 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:47:45.658 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:47:45.658 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:47:45.658 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:47:45.658 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:47:45.661 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:47:45.662 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:47:46.384 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 41 16 6309 6293 1.19902 1.15625 0.125939 0.104053 2024-04-23T01:47:46.769 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:47:46.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:46.889+0000 7ff993aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:46.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:46.897+0000 7ff993aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:47.384 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 42 16 6436 6420 1.19409 0.992188 0.219889 0.104493 2024-04-23T01:47:48.384 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 43 16 6576 6560 1.19176 1.09375 0.0328851 0.104652 2024-04-23T01:47:49.094 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:47:49.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:49.232+0000 7fadba35d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:49.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:49.239+0000 7fadba35d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:49.384 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 44 16 6705 6689 1.18758 1.00781 0.211288 0.105032 2024-04-23T01:47:50.384 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 45 16 6879 6863 1.19139 1.35938 0.0852094 0.104778 2024-04-23T01:47:50.662 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:47:50.662 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:47:50.662 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:47:50.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:50.785+0000 7f72e595d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:50.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:50.794+0000 7f72e595d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:51.082 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:47:51.083 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:47:51.099 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2024-04-23T01:47:49.579479+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":52437536,"num_objects":795,"num_object_clones":0,"num_object_copies":2385,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":6466,"num_write_kb":52526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6410,"ondisk_log_size":6410,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":266208,"kb_used_data":75648,"kb_used_omap":0,"kb_used_meta":190528,"kb_avail":377221152,"statfs":{"total":386547056640,"available":386274459648,"internally_reserved":0,"allocated":77463552,"data_stored":154688192,"data_compressed":7965750,"data_compressed_allocated":76419072,"data_compressed_original":154312704,"omap_allocated":0,"internal_metadata":195100672},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11575296,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1413,"num_write_kb":11304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002358"},"pg_stats":[{"pgid":"4.f","version":"29'416","reported_seq":429,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:45.528454+0000","last_change":"2024-04-23T01:47:04.491534+0000","last_active":"2024-04-23T01:47:45.528454+0000","last_peered":"2024-04-23T01:47:45.528454+0000","last_clean":"2024-04-23T01:47:45.528454+0000","last_became_active":"2024-04-23T01:47:04.486910+0000","last_became_peered":"2024-04-23T01:47:04.486910+0000","last_unstale":"2024-04-23T01:47:45.528454+0000","last_undegraded":"2024-04-23T01:47:45.528454+0000","last_fullsized":"2024-04-23T01:47:45.528454+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3416064,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":417,"num_write_kb":3336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"29'366","reported_seq":379,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:46.562453+0000","last_change":"2024-04-23T01:47:04.452063+0000","last_active":"2024-04-23T01:47:46.562453+0000","last_peered":"2024-04-23T01:47:46.562453+0000","last_clean":"2024-04-23T01:47:46.562453+0000","last_became_active":"2024-04-23T01:47:04.443666+0000","last_became_peered":"2024-04-23T01:47:04.443666+0000","last_unstale":"2024-04-23T01:47:46.562453+0000","last_undegraded":"2024-04-23T01:47:46.562453+0000","last_fullsized":"2024-04-23T01:47:46.562453+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3006464,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":367,"num_write_kb":2936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"29'391","reported_seq":404,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:42.339379+0000","last_change":"2024-04-23T01:47:04.485140+0000","last_active":"2024-04-23T01:47:42.339379+0000","last_peered":"2024-04-23T01:47:42.339379+0000","last_clean":"2024-04-23T01:47:42.339379+0000","last_became_active":"2024-04-23T01:47:04.480664+0000","last_became_peered":"2024-04-23T01:47:04.480664+0000","last_unstale":"2024-04-23T01:47:42.339379+0000","last_undegraded":"2024-04-23T01:47:42.339379+0000","last_fullsized":"2024-04-23T01:47:42.339379+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3211264,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":392,"num_write_kb":3136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"29'359","reported_seq":372,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:45.508405+0000","last_change":"2024-04-23T01:47:04.442089+0000","last_active":"2024-04-23T01:47:45.508405+0000","last_peered":"2024-04-23T01:47:45.508405+0000","last_clean":"2024-04-23T01:47:45.508405+0000","last_became_active":"2024-04-23T01:47:04.431317+0000","last_became_peered":"2024-04-23T01:47:04.431317+0000","last_unstale":"2024-04-23T01:47:45.508405+0000","last_undegraded":"2024-04-23T01:47:45.508405+0000","last_fullsized":"2024-04-23T01:47:45.508405+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2949120,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":360,"num_write_kb":2880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"29'415","reported_seq":428,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:44.082121+0000","last_change":"2024-04-23T01:47:04.503686+0000","last_active":"2024-04-23T01:47:44.082121+0000","last_peered":"2024-04-23T01:47:44.082121+0000","last_clean":"2024-04-23T01:47:44.082121+0000","last_became_active":"2024-04-23T01:47:04.499155+0000","last_became_peered":"2024-04-23T01:47:04.499155+0000","last_unstale":"2024-04-23T01:47:44.082121+0000","last_undegraded":"2024-04-23T01:47:44.082121+0000","last_fullsized":"2024-04-23T01:47:44.082121+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3407872,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":3328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"29'415","reported_seq":428,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:44.523349+0000","last_change":"2024-04-23T01:47:04.516672+0000","last_active":"2024-04-23T01:47:44.523349+0000","last_peered":"2024-04-23T01:47:44.523349+0000","last_clean":"2024-04-23T01:47:44.523349+0000","last_became_active":"2024-04-23T01:47:04.504711+0000","last_became_peered":"2024-04-23T01:47:04.504711+0000","last_unstale":"2024-04-23T01:47:44.523349+0000","last_undegraded":"2024-04-23T01:47:44.523349+0000","last_fullsized":"2024-04-23T01:47:44.523349+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3407872,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":3328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"29'343","reported_seq":356,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:46.275871+0000","last_change":"2024-04-23T01:47:04.449451+0000","last_active":"2024-04-23T01:47:46.275871+0000","last_peered":"2024-04-23T01:47:46.275871+0000","last_clean":"2024-04-23T01:47:46.275871+0000","last_became_active":"2024-04-23T01:47:04.442623+0000","last_became_peered":"2024-04-23T01:47:04.442623+0000","last_unstale":"2024-04-23T01:47:46.275871+0000","last_undegraded":"2024-04-23T01:47:46.275871+0000","last_fullsized":"2024-04-23T01:47:46.275871+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":344,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"29'404","reported_seq":417,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:46.558904+0000","last_change":"2024-04-23T01:47:04.520179+0000","last_active":"2024-04-23T01:47:46.558904+0000","last_peered":"2024-04-23T01:47:46.558904+0000","last_clean":"2024-04-23T01:47:46.558904+0000","last_became_active":"2024-04-23T01:47:04.511239+0000","last_became_peered":"2024-04-23T01:47:04.511239+0000","last_unstale":"2024-04-23T01:47:46.558904+0000","last_undegraded":"2024-04-23T01:47:46.558904+0000","last_fullsized":"2024-04-23T01:47:46.558904+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3317760,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":405,"num_write_kb":3240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"29'431","reported_seq":444,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:46.378251+0000","last_change":"2024-04-23T01:47:04.496716+0000","last_active":"2024-04-23T01:47:46.378251+0000","last_peered":"2024-04-23T01:47:46.378251+0000","last_clean":"2024-04-23T01:47:46.378251+0000","last_became_active":"2024-04-23T01:47:04.486607+0000","last_became_peered":"2024-04-23T01:47:04.486607+0000","last_unstale":"2024-04-23T01:47:46.378251+0000","last_undegraded":"2024-04-23T01:47:46.378251+0000","last_fullsized":"2024-04-23T01:47:46.378251+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3538944,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":432,"num_write_kb":3456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":186,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.325282+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:47:05.325282+0000","last_peered":"2024-04-23T01:47:05.325282+0000","last_clean":"2024-04-23T01:47:05.325282+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:47:05.325282+0000","last_undegraded":"2024-04-23T01:47:05.325282+0000","last_fullsized":"2024-04-23T01:47:05.325282+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.410555+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:47:05.410555+0000","last_peered":"2024-04-23T01:47:05.410555+0000","last_clean":"2024-04-23T01:47:05.410555+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:47:05.410555+0000","last_undegraded":"2024-04-23T01:47:05.410555+0000","last_fullsized":"2024-04-23T01:47:05.410555+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"29'319","reported_seq":332,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:46.551734+0000","last_change":"2024-04-23T01:47:04.467386+0000","last_active":"2024-04-23T01:47:46.551734+0000","last_peered":"2024-04-23T01:47:46.551734+0000","last_clean":"2024-04-23T01:47:46.551734+0000","last_became_active":"2024-04-23T01:47:04.459042+0000","last_became_peered":"2024-04-23T01:47:04.459042+0000","last_unstale":"2024-04-23T01:47:46.551734+0000","last_undegraded":"2024-04-23T01:47:46.551734+0000","last_fullsized":"2024-04-23T01:47:46.551734+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":320,"num_write_kb":2560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.570797+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:47:05.570797+0000","last_peered":"2024-04-23T01:47:05.570797+0000","last_clean":"2024-04-23T01:47:05.570797+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:47:05.570797+0000","last_undegraded":"2024-04-23T01:47:05.570797+0000","last_fullsized":"2024-04-23T01:47:05.570797+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"29'415","reported_seq":428,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:45.975864+0000","last_change":"2024-04-23T01:47:04.466464+0000","last_active":"2024-04-23T01:47:45.975864+0000","last_peered":"2024-04-23T01:47:45.975864+0000","last_clean":"2024-04-23T01:47:45.975864+0000","last_became_active":"2024-04-23T01:47:04.460338+0000","last_became_peered":"2024-04-23T01:47:04.460338+0000","last_unstale":"2024-04-23T01:47:45.975864+0000","last_undegraded":"2024-04-23T01:47:45.975864+0000","last_fullsized":"2024-04-23T01:47:45.975864+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3407872,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":3328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.397538+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:47:05.397538+0000","last_peered":"2024-04-23T01:47:05.397538+0000","last_clean":"2024-04-23T01:47:05.397538+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:47:05.397538+0000","last_undegraded":"2024-04-23T01:47:05.397538+0000","last_fullsized":"2024-04-23T01:47:05.397538+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"29'351","reported_seq":364,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:44.703835+0000","last_change":"2024-04-23T01:47:04.435560+0000","last_active":"2024-04-23T01:47:44.703835+0000","last_peered":"2024-04-23T01:47:44.703835+0000","last_clean":"2024-04-23T01:47:44.703835+0000","last_became_active":"2024-04-23T01:47:04.424493+0000","last_became_peered":"2024-04-23T01:47:04.424493+0000","last_unstale":"2024-04-23T01:47:44.703835+0000","last_undegraded":"2024-04-23T01:47:44.703835+0000","last_fullsized":"2024-04-23T01:47:44.703835+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883584,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":352,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.576933+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:47:05.576933+0000","last_peered":"2024-04-23T01:47:05.576933+0000","last_clean":"2024-04-23T01:47:05.576933+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:47:05.576933+0000","last_undegraded":"2024-04-23T01:47:05.576933+0000","last_fullsized":"2024-04-23T01:47:05.576933+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"29'415","reported_seq":428,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:46.459701+0000","last_change":"2024-04-23T01:47:04.530152+0000","last_active":"2024-04-23T01:47:46.459701+0000","last_peered":"2024-04-23T01:47:46.459701+0000","last_clean":"2024-04-23T01:47:46.459701+0000","last_became_active":"2024-04-23T01:47:04.519844+0000","last_became_peered":"2024-04-23T01:47:04.519844+0000","last_unstale":"2024-04-23T01:47:46.459701+0000","last_undegraded":"2024-04-23T01:47:46.459701+0000","last_fullsized":"2024-04-23T01:47:46.459701+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3407872,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":3328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.590971+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:47:05.590971+0000","last_peered":"2024-04-23T01:47:05.590971+0000","last_clean":"2024-04-23T01:47:05.590971+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:47:05.590971+0000","last_undegraded":"2024-04-23T01:47:05.590971+0000","last_fullsized":"2024-04-23T01:47:05.590971+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"29'455","reported_seq":468,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:45.751450+0000","last_change":"2024-04-23T01:47:04.465832+0000","last_active":"2024-04-23T01:47:45.751450+0000","last_peered":"2024-04-23T01:47:45.751450+0000","last_clean":"2024-04-23T01:47:45.751450+0000","last_became_active":"2024-04-23T01:47:04.460695+0000","last_became_peered":"2024-04-23T01:47:04.460695+0000","last_unstale":"2024-04-23T01:47:45.751450+0000","last_undegraded":"2024-04-23T01:47:45.751450+0000","last_fullsized":"2024-04-23T01:47:45.751450+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":456,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.331332+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:47:05.331332+0000","last_peered":"2024-04-23T01:47:05.331332+0000","last_clean":"2024-04-23T01:47:05.331332+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:47:05.331332+0000","last_undegraded":"2024-04-23T01:47:05.331332+0000","last_fullsized":"2024-04-23T01:47:05.331332+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"29'423","reported_seq":436,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:45.829530+0000","last_change":"2024-04-23T01:47:04.434787+0000","last_active":"2024-04-23T01:47:45.829530+0000","last_peered":"2024-04-23T01:47:45.829530+0000","last_clean":"2024-04-23T01:47:45.829530+0000","last_became_active":"2024-04-23T01:47:04.423352+0000","last_became_peered":"2024-04-23T01:47:04.423352+0000","last_unstale":"2024-04-23T01:47:45.829530+0000","last_undegraded":"2024-04-23T01:47:45.829530+0000","last_fullsized":"2024-04-23T01:47:45.829530+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3473408,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":424,"num_write_kb":3392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.602643+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:47:05.602643+0000","last_peered":"2024-04-23T01:47:05.602643+0000","last_clean":"2024-04-23T01:47:05.602643+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:47:05.602643+0000","last_undegraded":"2024-04-23T01:47:05.602643+0000","last_fullsized":"2024-04-23T01:47:05.602643+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"29'395","reported_seq":408,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:46.573484+0000","last_change":"2024-04-23T01:47:04.425726+0000","last_active":"2024-04-23T01:47:46.573484+0000","last_peered":"2024-04-23T01:47:46.573484+0000","last_clean":"2024-04-23T01:47:46.573484+0000","last_became_active":"2024-04-23T01:47:04.416905+0000","last_became_peered":"2024-04-23T01:47:04.416905+0000","last_unstale":"2024-04-23T01:47:46.573484+0000","last_undegraded":"2024-04-23T01:47:46.573484+0000","last_fullsized":"2024-04-23T01:47:46.573484+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3244032,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":396,"num_write_kb":3168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.400601+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:47:05.400601+0000","last_peered":"2024-04-23T01:47:05.400601+0000","last_clean":"2024-04-23T01:47:05.400601+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:47:05.400601+0000","last_undegraded":"2024-04-23T01:47:05.400601+0000","last_fullsized":"2024-04-23T01:47:05.400601+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":51847168,"num_objects":793,"num_object_clones":0,"num_object_copies":2379,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6329,"num_write_kb":50632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":77742080,"data_stored":155484160,"data_compressed":8025210,"data_compressed_allocated":77742080,"data_compressed_original":155484160,"omap_allocated":0,"internal_metadata":0},"log_size":6313,"ondisk_log_size":6313,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705823,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":73268,"kb_used_data":20076,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94298572,"statfs":{"total":96636764160,"available":96561737728,"internally_reserved":0,"allocated":20557824,"data_stored":41172792,"data_compressed":2120403,"data_compressed_allocated":20291584,"data_compressed_original":41074688,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705823,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50996,"kb_used_data":15852,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94320844,"statfs":{"total":96636764160,"available":96584544256,"internally_reserved":0,"allocated":16232448,"data_stored":32522040,"data_compressed":1673849,"data_compressed_allocated":15966208,"data_compressed_original":32423936,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705823,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65460,"kb_used_data":20588,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94306380,"statfs":{"total":96636764160,"available":96569733120,"internally_reserved":0,"allocated":21082112,"data_stored":41753880,"data_compressed":2150907,"data_compressed_allocated":20836352,"data_compressed_original":41672704,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705823,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":76484,"kb_used_data":19132,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94295356,"statfs":{"total":96636764160,"available":96558444544,"internally_reserved":0,"allocated":19591168,"data_stored":39239480,"data_compressed":2020591,"data_compressed_allocated":19324928,"data_compressed_original":39141376,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":19558400,"data_stored":39116800,"data_compressed":2018979,"data_compressed_allocated":19558400,"data_compressed_original":39116800,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":21200896,"data_stored":42401792,"data_compressed":2188554,"data_compressed_allocated":21200896,"data_compressed_original":42401792,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16252928,"data_stored":32505856,"data_compressed":1677736,"data_compressed_allocated":16252928,"data_compressed_original":32505856,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20729856,"data_stored":41459712,"data_compressed":2139941,"data_compressed_allocated":20729856,"data_compressed_original":41459712,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:47:51.384 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 46 16 7029 7013 1.19096 1.17188 0.0273993 0.104743 2024-04-23T01:47:51.448 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:47:51.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:51.569+0000 7fa8e3cad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:51.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:51.575+0000 7fa8e3cad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:52.384 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 47 16 7182 7166 1.19106 1.19531 0.145274 0.104788 2024-04-23T01:47:53.384 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 48 16 7340 7324 1.19196 1.23438 0.141945 0.104737 2024-04-23T01:47:53.791 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:47:53.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:53.905+0000 7f78c726d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:53.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:53.912+0000 7f78c726d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:54.384 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 49 16 7464 7448 1.1874 0.96875 0.227712 0.105118 2024-04-23T01:47:55.385 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 50 16 7612 7596 1.18677 1.15625 0.15429 0.105195 2024-04-23T01:47:56.100 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:47:56.101 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:47:56.101 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:47:56.101 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:47:56.138 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:47:56.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:56.247+0000 7fac92aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:56.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:56.261+0000 7fac92aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:56.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:56.280+0000 7f02cbc2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:56.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:56.287+0000 7f02cbc2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:56.288 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:47:56.288 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:47:56.288 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:47:56.288 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:47:56.288 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:47:56.288 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:47:56.288 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:47:56.289 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:47:56.289 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:47:56.289 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:47:56.289 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:47:56.289 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:47:56.292 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:47:56.293 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:47:56.293 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:47:56.385 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 51 16 7761 7745 1.18633 1.16406 0.154686 0.105191 2024-04-23T01:47:56.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:56.412+0000 7ff01a6ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:56.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:56.422+0000 7ff01a6ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:56.532 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:47:56.532 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:47:56.532 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:47:56.532 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:47:56.532 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:47:56.532 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:47:56.532 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:47:56.532 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:47:56.532 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:47:56.532 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:47:56.532 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:47:56.532 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:47:56.534 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:47:56.535 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:47:56.535 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:47:56.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:56.700+0000 7f2b67b9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:56.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:56.712+0000 7f2b67b9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:56.743 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:47:56.743 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:47:56.743 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:47:56.743 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:47:56.743 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:47:56.743 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:47:56.743 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:47:56.744 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:47:56.744 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:47:56.744 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:47:56.744 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:47:56.744 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:47:56.748 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:47:56.749 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:47:56.749 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:47:56.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:56.873+0000 7ff0bc46d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:56.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:56.883+0000 7ff0bc46d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:56.904 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:47:56.905 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:47:56.905 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:47:56.905 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:47:56.905 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:47:56.905 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:47:56.905 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:47:56.905 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:47:56.905 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:47:56.905 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:47:56.905 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:47:56.905 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:47:56.909 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:47:56.909 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:47:57.386 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 52 16 7891 7875 1.18304 1.01562 0.115974 0.105472 2024-04-23T01:47:58.385 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 53 16 8038 8022 1.18239 1.14844 0.0643718 0.105611 2024-04-23T01:47:58.505 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:47:58.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:58.642+0000 7f9362d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:58.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:47:58.649+0000 7f9362d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:47:59.385 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 54 16 8181 8165 1.18118 1.11719 0.0900692 0.105711 2024-04-23T01:48:00.385 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 55 16 8319 8303 1.1793 1.07812 0.0567084 0.105886 2024-04-23T01:48:00.791 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:48:00.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:00.908+0000 7f9f3e26d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:00.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:00.917+0000 7f9f3e26d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:01.385 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 56 16 8449 8433 1.17638 1.01562 0.151943 0.106078 2024-04-23T01:48:01.910 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:48:01.911 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:48:01.911 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:48:02.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:02.046+0000 7f3e15e2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:02.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:02.058+0000 7f3e15e2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:02.376 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:48:02.377 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:48:02.385 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 57 16 8599 8583 1.1763 1.17188 0.189739 0.106114 2024-04-23T01:48:02.397 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2024-04-23T01:48:01.581747+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":64225824,"num_objects":975,"num_object_clones":0,"num_object_copies":2925,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":7905,"num_write_kb":64038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7849,"ondisk_log_size":7849,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":301672,"kb_used_data":93056,"kb_used_omap":0,"kb_used_meta":208576,"kb_avail":377185688,"statfs":{"total":386547056640,"available":386238144512,"internally_reserved":0,"allocated":95289344,"data_stored":190339776,"data_compressed":9806646,"data_compressed_allocated":94244864,"data_compressed_original":189964288,"omap_allocated":0,"internal_metadata":213581824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11788288,"num_objects":180,"num_object_clones":0,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1439,"num_write_kb":11512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002268"},"pg_stats":[{"pgid":"4.f","version":"29'487","reported_seq":500,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:56.003315+0000","last_change":"2024-04-23T01:47:04.491534+0000","last_active":"2024-04-23T01:47:56.003315+0000","last_peered":"2024-04-23T01:47:56.003315+0000","last_clean":"2024-04-23T01:47:56.003315+0000","last_became_active":"2024-04-23T01:47:04.486910+0000","last_became_peered":"2024-04-23T01:47:04.486910+0000","last_unstale":"2024-04-23T01:47:56.003315+0000","last_undegraded":"2024-04-23T01:47:56.003315+0000","last_fullsized":"2024-04-23T01:47:56.003315+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3997696,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":488,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"29'423","reported_seq":436,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:55.686661+0000","last_change":"2024-04-23T01:47:04.452063+0000","last_active":"2024-04-23T01:47:55.686661+0000","last_peered":"2024-04-23T01:47:55.686661+0000","last_clean":"2024-04-23T01:47:55.686661+0000","last_became_active":"2024-04-23T01:47:04.443666+0000","last_became_peered":"2024-04-23T01:47:04.443666+0000","last_unstale":"2024-04-23T01:47:55.686661+0000","last_undegraded":"2024-04-23T01:47:55.686661+0000","last_fullsized":"2024-04-23T01:47:55.686661+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3473408,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":424,"num_write_kb":3392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"29'471","reported_seq":484,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:56.129526+0000","last_change":"2024-04-23T01:47:04.485140+0000","last_active":"2024-04-23T01:47:56.129526+0000","last_peered":"2024-04-23T01:47:56.129526+0000","last_clean":"2024-04-23T01:47:56.129526+0000","last_became_active":"2024-04-23T01:47:04.480664+0000","last_became_peered":"2024-04-23T01:47:04.480664+0000","last_unstale":"2024-04-23T01:47:56.129526+0000","last_undegraded":"2024-04-23T01:47:56.129526+0000","last_fullsized":"2024-04-23T01:47:56.129526+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3866624,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":472,"num_write_kb":3776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"29'483","reported_seq":496,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:56.547345+0000","last_change":"2024-04-23T01:47:04.442089+0000","last_active":"2024-04-23T01:47:56.547345+0000","last_peered":"2024-04-23T01:47:56.547345+0000","last_clean":"2024-04-23T01:47:56.547345+0000","last_became_active":"2024-04-23T01:47:04.431317+0000","last_became_peered":"2024-04-23T01:47:04.431317+0000","last_unstale":"2024-04-23T01:47:56.547345+0000","last_undegraded":"2024-04-23T01:47:56.547345+0000","last_fullsized":"2024-04-23T01:47:56.547345+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3964928,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":484,"num_write_kb":3872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"29'503","reported_seq":516,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:56.025063+0000","last_change":"2024-04-23T01:47:04.503686+0000","last_active":"2024-04-23T01:47:56.025063+0000","last_peered":"2024-04-23T01:47:56.025063+0000","last_clean":"2024-04-23T01:47:56.025063+0000","last_became_active":"2024-04-23T01:47:04.499155+0000","last_became_peered":"2024-04-23T01:47:04.499155+0000","last_unstale":"2024-04-23T01:47:56.025063+0000","last_undegraded":"2024-04-23T01:47:56.025063+0000","last_fullsized":"2024-04-23T01:47:56.025063+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4128768,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":504,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"29'543","reported_seq":556,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:55.386691+0000","last_change":"2024-04-23T01:47:04.516672+0000","last_active":"2024-04-23T01:47:55.386691+0000","last_peered":"2024-04-23T01:47:55.386691+0000","last_clean":"2024-04-23T01:47:55.386691+0000","last_became_active":"2024-04-23T01:47:04.504711+0000","last_became_peered":"2024-04-23T01:47:04.504711+0000","last_unstale":"2024-04-23T01:47:55.386691+0000","last_undegraded":"2024-04-23T01:47:55.386691+0000","last_fullsized":"2024-04-23T01:47:55.386691+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":544,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"29'479","reported_seq":492,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:56.466264+0000","last_change":"2024-04-23T01:47:04.449451+0000","last_active":"2024-04-23T01:47:56.466264+0000","last_peered":"2024-04-23T01:47:56.466264+0000","last_clean":"2024-04-23T01:47:56.466264+0000","last_became_active":"2024-04-23T01:47:04.442623+0000","last_became_peered":"2024-04-23T01:47:04.442623+0000","last_unstale":"2024-04-23T01:47:56.466264+0000","last_undegraded":"2024-04-23T01:47:56.466264+0000","last_fullsized":"2024-04-23T01:47:56.466264+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":480,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"29'487","reported_seq":500,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:56.318912+0000","last_change":"2024-04-23T01:47:04.520179+0000","last_active":"2024-04-23T01:47:56.318912+0000","last_peered":"2024-04-23T01:47:56.318912+0000","last_clean":"2024-04-23T01:47:56.318912+0000","last_became_active":"2024-04-23T01:47:04.511239+0000","last_became_peered":"2024-04-23T01:47:04.511239+0000","last_unstale":"2024-04-23T01:47:56.318912+0000","last_undegraded":"2024-04-23T01:47:56.318912+0000","last_fullsized":"2024-04-23T01:47:56.318912+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3997696,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":488,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"29'522","reported_seq":535,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:56.583354+0000","last_change":"2024-04-23T01:47:04.496716+0000","last_active":"2024-04-23T01:47:56.583354+0000","last_peered":"2024-04-23T01:47:56.583354+0000","last_clean":"2024-04-23T01:47:56.583354+0000","last_became_active":"2024-04-23T01:47:04.486607+0000","last_became_peered":"2024-04-23T01:47:04.486607+0000","last_unstale":"2024-04-23T01:47:56.583354+0000","last_undegraded":"2024-04-23T01:47:56.583354+0000","last_fullsized":"2024-04-23T01:47:56.583354+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":522,"log_dups_size":0,"ondisk_log_size":522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4284416,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":523,"num_write_kb":4184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":186,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.325282+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:47:05.325282+0000","last_peered":"2024-04-23T01:47:05.325282+0000","last_clean":"2024-04-23T01:47:05.325282+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:47:05.325282+0000","last_undegraded":"2024-04-23T01:47:05.325282+0000","last_fullsized":"2024-04-23T01:47:05.325282+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.410555+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:47:05.410555+0000","last_peered":"2024-04-23T01:47:05.410555+0000","last_clean":"2024-04-23T01:47:05.410555+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:47:05.410555+0000","last_undegraded":"2024-04-23T01:47:05.410555+0000","last_fullsized":"2024-04-23T01:47:05.410555+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"29'439","reported_seq":452,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:56.256585+0000","last_change":"2024-04-23T01:47:04.467386+0000","last_active":"2024-04-23T01:47:56.256585+0000","last_peered":"2024-04-23T01:47:56.256585+0000","last_clean":"2024-04-23T01:47:56.256585+0000","last_became_active":"2024-04-23T01:47:04.459042+0000","last_became_peered":"2024-04-23T01:47:04.459042+0000","last_unstale":"2024-04-23T01:47:56.256585+0000","last_undegraded":"2024-04-23T01:47:56.256585+0000","last_fullsized":"2024-04-23T01:47:56.256585+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3604480,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":440,"num_write_kb":3520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.570797+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:47:05.570797+0000","last_peered":"2024-04-23T01:47:05.570797+0000","last_clean":"2024-04-23T01:47:05.570797+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:47:05.570797+0000","last_undegraded":"2024-04-23T01:47:05.570797+0000","last_fullsized":"2024-04-23T01:47:05.570797+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"29'472","reported_seq":485,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:55.025789+0000","last_change":"2024-04-23T01:47:04.466464+0000","last_active":"2024-04-23T01:47:55.025789+0000","last_peered":"2024-04-23T01:47:55.025789+0000","last_clean":"2024-04-23T01:47:55.025789+0000","last_became_active":"2024-04-23T01:47:04.460338+0000","last_became_peered":"2024-04-23T01:47:04.460338+0000","last_unstale":"2024-04-23T01:47:55.025789+0000","last_undegraded":"2024-04-23T01:47:55.025789+0000","last_fullsized":"2024-04-23T01:47:55.025789+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3874816,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":473,"num_write_kb":3784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.397538+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:47:05.397538+0000","last_peered":"2024-04-23T01:47:05.397538+0000","last_clean":"2024-04-23T01:47:05.397538+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:47:05.397538+0000","last_undegraded":"2024-04-23T01:47:05.397538+0000","last_fullsized":"2024-04-23T01:47:05.397538+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"29'471","reported_seq":484,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:56.093062+0000","last_change":"2024-04-23T01:47:04.435560+0000","last_active":"2024-04-23T01:47:56.093062+0000","last_peered":"2024-04-23T01:47:56.093062+0000","last_clean":"2024-04-23T01:47:56.093062+0000","last_became_active":"2024-04-23T01:47:04.424493+0000","last_became_peered":"2024-04-23T01:47:04.424493+0000","last_unstale":"2024-04-23T01:47:56.093062+0000","last_undegraded":"2024-04-23T01:47:56.093062+0000","last_fullsized":"2024-04-23T01:47:56.093062+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3866624,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":472,"num_write_kb":3776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.576933+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:47:05.576933+0000","last_peered":"2024-04-23T01:47:05.576933+0000","last_clean":"2024-04-23T01:47:05.576933+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:47:05.576933+0000","last_undegraded":"2024-04-23T01:47:05.576933+0000","last_fullsized":"2024-04-23T01:47:05.576933+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"29'471","reported_seq":484,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:56.327189+0000","last_change":"2024-04-23T01:47:04.530152+0000","last_active":"2024-04-23T01:47:56.327189+0000","last_peered":"2024-04-23T01:47:56.327189+0000","last_clean":"2024-04-23T01:47:56.327189+0000","last_became_active":"2024-04-23T01:47:04.519844+0000","last_became_peered":"2024-04-23T01:47:04.519844+0000","last_unstale":"2024-04-23T01:47:56.327189+0000","last_undegraded":"2024-04-23T01:47:56.327189+0000","last_fullsized":"2024-04-23T01:47:56.327189+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3866624,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":472,"num_write_kb":3776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.590971+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:47:05.590971+0000","last_peered":"2024-04-23T01:47:05.590971+0000","last_clean":"2024-04-23T01:47:05.590971+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:47:05.590971+0000","last_undegraded":"2024-04-23T01:47:05.590971+0000","last_fullsized":"2024-04-23T01:47:05.590971+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"29'559","reported_seq":572,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:54.576761+0000","last_change":"2024-04-23T01:47:04.465832+0000","last_active":"2024-04-23T01:47:54.576761+0000","last_peered":"2024-04-23T01:47:54.576761+0000","last_clean":"2024-04-23T01:47:54.576761+0000","last_became_active":"2024-04-23T01:47:04.460695+0000","last_became_peered":"2024-04-23T01:47:04.460695+0000","last_unstale":"2024-04-23T01:47:54.576761+0000","last_undegraded":"2024-04-23T01:47:54.576761+0000","last_fullsized":"2024-04-23T01:47:54.576761+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4587520,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":560,"num_write_kb":4480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.331332+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:47:05.331332+0000","last_peered":"2024-04-23T01:47:05.331332+0000","last_clean":"2024-04-23T01:47:05.331332+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:47:05.331332+0000","last_undegraded":"2024-04-23T01:47:05.331332+0000","last_fullsized":"2024-04-23T01:47:05.331332+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"29'503","reported_seq":516,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:55.484915+0000","last_change":"2024-04-23T01:47:04.434787+0000","last_active":"2024-04-23T01:47:55.484915+0000","last_peered":"2024-04-23T01:47:55.484915+0000","last_clean":"2024-04-23T01:47:55.484915+0000","last_became_active":"2024-04-23T01:47:04.423352+0000","last_became_peered":"2024-04-23T01:47:04.423352+0000","last_unstale":"2024-04-23T01:47:55.484915+0000","last_undegraded":"2024-04-23T01:47:55.484915+0000","last_fullsized":"2024-04-23T01:47:55.484915+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4128768,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":504,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.602643+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:47:05.602643+0000","last_peered":"2024-04-23T01:47:05.602643+0000","last_clean":"2024-04-23T01:47:05.602643+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:47:05.602643+0000","last_undegraded":"2024-04-23T01:47:05.602643+0000","last_fullsized":"2024-04-23T01:47:05.602643+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"29'439","reported_seq":452,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:55.149496+0000","last_change":"2024-04-23T01:47:04.425726+0000","last_active":"2024-04-23T01:47:55.149496+0000","last_peered":"2024-04-23T01:47:55.149496+0000","last_clean":"2024-04-23T01:47:55.149496+0000","last_became_active":"2024-04-23T01:47:04.416905+0000","last_became_peered":"2024-04-23T01:47:04.416905+0000","last_unstale":"2024-04-23T01:47:55.149496+0000","last_undegraded":"2024-04-23T01:47:55.149496+0000","last_fullsized":"2024-04-23T01:47:55.149496+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3604480,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":440,"num_write_kb":3520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.400601+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:47:05.400601+0000","last_peered":"2024-04-23T01:47:05.400601+0000","last_clean":"2024-04-23T01:47:05.400601+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:47:05.400601+0000","last_undegraded":"2024-04-23T01:47:05.400601+0000","last_fullsized":"2024-04-23T01:47:05.400601+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":63635456,"num_objects":973,"num_object_clones":0,"num_object_copies":2919,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7768,"num_write_kb":62144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":95457280,"data_stored":190914560,"data_compressed":9854685,"data_compressed_allocated":95457280,"data_compressed_original":190914560,"omap_allocated":0,"internal_metadata":0},"log_size":7752,"ondisk_log_size":7752,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705833,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":82236,"kb_used_data":24880,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94289604,"statfs":{"total":96636764160,"available":96552554496,"internally_reserved":0,"allocated":25477120,"data_stored":51011384,"data_compressed":2628426,"data_compressed_allocated":25210880,"data_compressed_original":50913280,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705833,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":76744,"kb_used_data":19392,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94295096,"statfs":{"total":96636764160,"available":96558178304,"internally_reserved":0,"allocated":19857408,"data_stored":39771960,"data_compressed":2048204,"data_compressed_allocated":19591168,"data_compressed_original":39673856,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705833,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":74396,"kb_used_data":25364,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94297444,"statfs":{"total":96636764160,"available":96560582656,"internally_reserved":0,"allocated":25972736,"data_stored":51535128,"data_compressed":2655969,"data_compressed_allocated":25726976,"data_compressed_original":51453952,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705833,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":68296,"kb_used_data":23420,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94303544,"statfs":{"total":96636764160,"available":96566829056,"internally_reserved":0,"allocated":23982080,"data_stored":48021304,"data_compressed":2474047,"data_compressed_allocated":23715840,"data_compressed_original":47923200,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":23982080,"data_stored":47964160,"data_compressed":2475819,"data_compressed_allocated":23982080,"data_compressed_original":47964160,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26083328,"data_stored":52166656,"data_compressed":2692770,"data_compressed_allocated":26083328,"data_compressed_original":52166656,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19865600,"data_stored":39731200,"data_compressed":2050822,"data_compressed_allocated":19865600,"data_compressed_original":39731200,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":25526272,"data_stored":51052544,"data_compressed":2635274,"data_compressed_allocated":25526272,"data_compressed_original":51052544,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:48:03.174 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:48:03.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:03.299+0000 7ff2b7e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:03.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:03.305+0000 7ff2b7e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:03.385 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 58 16 8749 8733 1.17622 1.17188 0.153579 0.106159 2024-04-23T01:48:04.385 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 59 16 8880 8864 1.17363 1.02344 0.138278 0.106361 2024-04-23T01:48:05.386 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:48:05.387282+0000 min lat: 0.0142109 max lat: 0.372373 avg lat: 0.106318 2024-04-23T01:48:05.386 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:48:05.386 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 60 16 9034 9018 1.17412 1.20312 0.0491093 0.106318 2024-04-23T01:48:05.535 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:48:05.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:05.680+0000 7fe220206640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:05.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:05.689+0000 7fe220206640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:06.386 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 61 16 9175 9159 1.17293 1.10156 0.114652 0.106419 2024-04-23T01:48:07.386 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 62 16 9299 9283 1.16963 0.96875 0.0640896 0.106723 2024-04-23T01:48:07.400 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:48:07.400 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:48:07.401 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:48:07.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:07.534+0000 7fd12039d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:07.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:07.546+0000 7fd12039d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:07.769 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:48:07.770 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:48:07.783 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-04-23T01:48:07.582856+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":75866656,"num_objects":1152,"num_object_clones":0,"num_object_copies":3456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":9326,"num_write_kb":75406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9270,"ondisk_log_size":9270,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":318740,"kb_used_data":110132,"kb_used_omap":0,"kb_used_meta":208576,"kb_avail":377168620,"statfs":{"total":386547056640,"available":386220666880,"internally_reserved":0,"allocated":112775168,"data_stored":225311424,"data_compressed":11612433,"data_compressed_allocated":111730688,"data_compressed_original":224935936,"omap_allocated":0,"internal_metadata":213581824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17539072,"num_objects":267,"num_object_clones":0,"num_object_copies":801,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2141,"num_write_kb":17128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002228"},"pg_stats":[{"pgid":"4.f","version":"29'575","reported_seq":588,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:05.393774+0000","last_change":"2024-04-23T01:47:04.491534+0000","last_active":"2024-04-23T01:48:05.393774+0000","last_peered":"2024-04-23T01:48:05.393774+0000","last_clean":"2024-04-23T01:48:05.393774+0000","last_became_active":"2024-04-23T01:47:04.486910+0000","last_became_peered":"2024-04-23T01:47:04.486910+0000","last_unstale":"2024-04-23T01:48:05.393774+0000","last_undegraded":"2024-04-23T01:48:05.393774+0000","last_fullsized":"2024-04-23T01:48:05.393774+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718592,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":576,"num_write_kb":4608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"29'511","reported_seq":524,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:04.435648+0000","last_change":"2024-04-23T01:47:04.452063+0000","last_active":"2024-04-23T01:48:04.435648+0000","last_peered":"2024-04-23T01:48:04.435648+0000","last_clean":"2024-04-23T01:48:04.435648+0000","last_became_active":"2024-04-23T01:47:04.443666+0000","last_became_peered":"2024-04-23T01:47:04.443666+0000","last_unstale":"2024-04-23T01:48:04.435648+0000","last_undegraded":"2024-04-23T01:48:04.435648+0000","last_fullsized":"2024-04-23T01:48:04.435648+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":512,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"29'543","reported_seq":556,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:05.465106+0000","last_change":"2024-04-23T01:47:04.485140+0000","last_active":"2024-04-23T01:48:05.465106+0000","last_peered":"2024-04-23T01:48:05.465106+0000","last_clean":"2024-04-23T01:48:05.465106+0000","last_became_active":"2024-04-23T01:47:04.480664+0000","last_became_peered":"2024-04-23T01:47:04.480664+0000","last_unstale":"2024-04-23T01:48:05.465106+0000","last_undegraded":"2024-04-23T01:48:05.465106+0000","last_fullsized":"2024-04-23T01:48:05.465106+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":544,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"29'559","reported_seq":572,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:05.771594+0000","last_change":"2024-04-23T01:47:04.442089+0000","last_active":"2024-04-23T01:48:05.771594+0000","last_peered":"2024-04-23T01:48:05.771594+0000","last_clean":"2024-04-23T01:48:05.771594+0000","last_became_active":"2024-04-23T01:47:04.431317+0000","last_became_peered":"2024-04-23T01:47:04.431317+0000","last_unstale":"2024-04-23T01:48:05.771594+0000","last_undegraded":"2024-04-23T01:48:05.771594+0000","last_fullsized":"2024-04-23T01:48:05.771594+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4587520,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":560,"num_write_kb":4480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"29'559","reported_seq":572,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:05.491475+0000","last_change":"2024-04-23T01:47:04.503686+0000","last_active":"2024-04-23T01:48:05.491475+0000","last_peered":"2024-04-23T01:48:05.491475+0000","last_clean":"2024-04-23T01:48:05.491475+0000","last_became_active":"2024-04-23T01:47:04.499155+0000","last_became_peered":"2024-04-23T01:47:04.499155+0000","last_unstale":"2024-04-23T01:48:05.491475+0000","last_undegraded":"2024-04-23T01:48:05.491475+0000","last_fullsized":"2024-04-23T01:48:05.491475+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4587520,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":560,"num_write_kb":4480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"29'663","reported_seq":676,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:06.275417+0000","last_change":"2024-04-23T01:47:04.516672+0000","last_active":"2024-04-23T01:48:06.275417+0000","last_peered":"2024-04-23T01:48:06.275417+0000","last_clean":"2024-04-23T01:48:06.275417+0000","last_became_active":"2024-04-23T01:47:04.504711+0000","last_became_peered":"2024-04-23T01:47:04.504711+0000","last_unstale":"2024-04-23T01:48:06.275417+0000","last_undegraded":"2024-04-23T01:48:06.275417+0000","last_fullsized":"2024-04-23T01:48:06.275417+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5439488,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":664,"num_write_kb":5312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"29'527","reported_seq":540,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:06.174653+0000","last_change":"2024-04-23T01:47:04.449451+0000","last_active":"2024-04-23T01:48:06.174653+0000","last_peered":"2024-04-23T01:48:06.174653+0000","last_clean":"2024-04-23T01:48:06.174653+0000","last_became_active":"2024-04-23T01:47:04.442623+0000","last_became_peered":"2024-04-23T01:47:04.442623+0000","last_unstale":"2024-04-23T01:48:06.174653+0000","last_undegraded":"2024-04-23T01:48:06.174653+0000","last_fullsized":"2024-04-23T01:48:06.174653+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":528,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"29'599","reported_seq":612,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:06.365653+0000","last_change":"2024-04-23T01:47:04.520179+0000","last_active":"2024-04-23T01:48:06.365653+0000","last_peered":"2024-04-23T01:48:06.365653+0000","last_clean":"2024-04-23T01:48:06.365653+0000","last_became_active":"2024-04-23T01:47:04.511239+0000","last_became_peered":"2024-04-23T01:47:04.511239+0000","last_unstale":"2024-04-23T01:48:06.365653+0000","last_undegraded":"2024-04-23T01:48:06.365653+0000","last_fullsized":"2024-04-23T01:48:06.365653+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"29'618","reported_seq":631,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:06.584808+0000","last_change":"2024-04-23T01:47:04.496716+0000","last_active":"2024-04-23T01:48:06.584808+0000","last_peered":"2024-04-23T01:48:06.584808+0000","last_clean":"2024-04-23T01:48:06.584808+0000","last_became_active":"2024-04-23T01:47:04.486607+0000","last_became_peered":"2024-04-23T01:47:04.486607+0000","last_unstale":"2024-04-23T01:48:06.584808+0000","last_undegraded":"2024-04-23T01:48:06.584808+0000","last_fullsized":"2024-04-23T01:48:06.584808+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5070848,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":619,"num_write_kb":4952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":186,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.325282+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:47:05.325282+0000","last_peered":"2024-04-23T01:47:05.325282+0000","last_clean":"2024-04-23T01:47:05.325282+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:47:05.325282+0000","last_undegraded":"2024-04-23T01:47:05.325282+0000","last_fullsized":"2024-04-23T01:47:05.325282+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.410555+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:47:05.410555+0000","last_peered":"2024-04-23T01:47:05.410555+0000","last_clean":"2024-04-23T01:47:05.410555+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:47:05.410555+0000","last_undegraded":"2024-04-23T01:47:05.410555+0000","last_fullsized":"2024-04-23T01:47:05.410555+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"29'519","reported_seq":532,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:06.218807+0000","last_change":"2024-04-23T01:47:04.467386+0000","last_active":"2024-04-23T01:48:06.218807+0000","last_peered":"2024-04-23T01:48:06.218807+0000","last_clean":"2024-04-23T01:48:06.218807+0000","last_became_active":"2024-04-23T01:47:04.459042+0000","last_became_peered":"2024-04-23T01:47:04.459042+0000","last_unstale":"2024-04-23T01:48:06.218807+0000","last_undegraded":"2024-04-23T01:48:06.218807+0000","last_fullsized":"2024-04-23T01:48:06.218807+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259840,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":520,"num_write_kb":4160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.570797+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:47:05.570797+0000","last_peered":"2024-04-23T01:47:05.570797+0000","last_clean":"2024-04-23T01:47:05.570797+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:47:05.570797+0000","last_undegraded":"2024-04-23T01:47:05.570797+0000","last_fullsized":"2024-04-23T01:47:05.570797+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"29'562","reported_seq":575,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:06.564389+0000","last_change":"2024-04-23T01:47:04.466464+0000","last_active":"2024-04-23T01:48:06.564389+0000","last_peered":"2024-04-23T01:48:06.564389+0000","last_clean":"2024-04-23T01:48:06.564389+0000","last_became_active":"2024-04-23T01:47:04.460338+0000","last_became_peered":"2024-04-23T01:47:04.460338+0000","last_unstale":"2024-04-23T01:48:06.564389+0000","last_undegraded":"2024-04-23T01:48:06.564389+0000","last_fullsized":"2024-04-23T01:48:06.564389+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4612096,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":563,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.397538+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:47:05.397538+0000","last_peered":"2024-04-23T01:47:05.397538+0000","last_clean":"2024-04-23T01:47:05.397538+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:47:05.397538+0000","last_undegraded":"2024-04-23T01:47:05.397538+0000","last_fullsized":"2024-04-23T01:47:05.397538+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"29'607","reported_seq":620,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:05.627990+0000","last_change":"2024-04-23T01:47:04.435560+0000","last_active":"2024-04-23T01:48:05.627990+0000","last_peered":"2024-04-23T01:48:05.627990+0000","last_clean":"2024-04-23T01:48:05.627990+0000","last_became_active":"2024-04-23T01:47:04.424493+0000","last_became_peered":"2024-04-23T01:47:04.424493+0000","last_unstale":"2024-04-23T01:48:05.627990+0000","last_undegraded":"2024-04-23T01:48:05.627990+0000","last_fullsized":"2024-04-23T01:48:05.627990+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4980736,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":608,"num_write_kb":4864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.576933+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:47:05.576933+0000","last_peered":"2024-04-23T01:47:05.576933+0000","last_clean":"2024-04-23T01:47:05.576933+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:47:05.576933+0000","last_undegraded":"2024-04-23T01:47:05.576933+0000","last_fullsized":"2024-04-23T01:47:05.576933+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"29'567","reported_seq":580,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:06.466436+0000","last_change":"2024-04-23T01:47:04.530152+0000","last_active":"2024-04-23T01:48:06.466436+0000","last_peered":"2024-04-23T01:48:06.466436+0000","last_clean":"2024-04-23T01:48:06.466436+0000","last_became_active":"2024-04-23T01:47:04.519844+0000","last_became_peered":"2024-04-23T01:47:04.519844+0000","last_unstale":"2024-04-23T01:48:06.466436+0000","last_undegraded":"2024-04-23T01:48:06.466436+0000","last_fullsized":"2024-04-23T01:48:06.466436+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":568,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.590971+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:47:05.590971+0000","last_peered":"2024-04-23T01:47:05.590971+0000","last_clean":"2024-04-23T01:47:05.590971+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:47:05.590971+0000","last_undegraded":"2024-04-23T01:47:05.590971+0000","last_fullsized":"2024-04-23T01:47:05.590971+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"29'662","reported_seq":675,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:06.549318+0000","last_change":"2024-04-23T01:47:04.465832+0000","last_active":"2024-04-23T01:48:06.549318+0000","last_peered":"2024-04-23T01:48:06.549318+0000","last_clean":"2024-04-23T01:48:06.549318+0000","last_became_active":"2024-04-23T01:47:04.460695+0000","last_became_peered":"2024-04-23T01:47:04.460695+0000","last_unstale":"2024-04-23T01:48:06.549318+0000","last_undegraded":"2024-04-23T01:48:06.549318+0000","last_fullsized":"2024-04-23T01:48:06.549318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5431296,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":663,"num_write_kb":5304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.331332+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:47:05.331332+0000","last_peered":"2024-04-23T01:47:05.331332+0000","last_clean":"2024-04-23T01:47:05.331332+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:47:05.331332+0000","last_undegraded":"2024-04-23T01:47:05.331332+0000","last_fullsized":"2024-04-23T01:47:05.331332+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"29'591","reported_seq":604,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:05.185395+0000","last_change":"2024-04-23T01:47:04.434787+0000","last_active":"2024-04-23T01:48:05.185395+0000","last_peered":"2024-04-23T01:48:05.185395+0000","last_clean":"2024-04-23T01:48:05.185395+0000","last_became_active":"2024-04-23T01:47:04.423352+0000","last_became_peered":"2024-04-23T01:47:04.423352+0000","last_unstale":"2024-04-23T01:48:05.185395+0000","last_undegraded":"2024-04-23T01:48:05.185395+0000","last_fullsized":"2024-04-23T01:48:05.185395+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":592,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.602643+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:47:05.602643+0000","last_peered":"2024-04-23T01:47:05.602643+0000","last_clean":"2024-04-23T01:47:05.602643+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:47:05.602643+0000","last_undegraded":"2024-04-23T01:47:05.602643+0000","last_fullsized":"2024-04-23T01:47:05.602643+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"29'511","reported_seq":524,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:02.068547+0000","last_change":"2024-04-23T01:47:04.425726+0000","last_active":"2024-04-23T01:48:02.068547+0000","last_peered":"2024-04-23T01:48:02.068547+0000","last_clean":"2024-04-23T01:48:02.068547+0000","last_became_active":"2024-04-23T01:47:04.416905+0000","last_became_peered":"2024-04-23T01:47:04.416905+0000","last_unstale":"2024-04-23T01:48:02.068547+0000","last_undegraded":"2024-04-23T01:48:02.068547+0000","last_fullsized":"2024-04-23T01:48:02.068547+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":512,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.400601+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:47:05.400601+0000","last_peered":"2024-04-23T01:47:05.400601+0000","last_clean":"2024-04-23T01:47:05.400601+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:47:05.400601+0000","last_undegraded":"2024-04-23T01:47:05.400601+0000","last_fullsized":"2024-04-23T01:47:05.400601+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":75276288,"num_objects":1150,"num_object_clones":0,"num_object_copies":3450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9189,"num_write_kb":73512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":112906240,"data_stored":225812480,"data_compressed":11656665,"data_compressed_allocated":112906240,"data_compressed_original":225812480,"omap_allocated":0,"internal_metadata":0},"log_size":9173,"ondisk_log_size":9173,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705843,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":74524,"kb_used_data":29652,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94297316,"statfs":{"total":96636764160,"available":96560451584,"internally_reserved":0,"allocated":30363648,"data_stored":60784440,"data_compressed":3133065,"data_compressed_allocated":30097408,"data_compressed_original":60686336,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705843,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":84656,"kb_used_data":23144,"kb_used_omap":0,"kb_used_meta":61504,"kb_avail":94287184,"statfs":{"total":96636764160,"available":96550076416,"internally_reserved":0,"allocated":23699456,"data_stored":47456056,"data_compressed":2444978,"data_compressed_allocated":23433216,"data_compressed_original":47357952,"omap_allocated":0,"internal_metadata":62980096},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705843,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":83176,"kb_used_data":29984,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94288664,"statfs":{"total":96636764160,"available":96551591936,"internally_reserved":0,"allocated":30703616,"data_stored":60996888,"data_compressed":3144534,"data_compressed_allocated":30457856,"data_compressed_original":60915712,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705843,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":76384,"kb_used_data":27352,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94295456,"statfs":{"total":96636764160,"available":96558546944,"internally_reserved":0,"allocated":28008448,"data_stored":56074040,"data_compressed":2889856,"data_compressed_allocated":27742208,"data_compressed_original":55975936,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":27996160,"data_stored":55992320,"data_compressed":2890359,"data_compressed_allocated":27996160,"data_compressed_original":55992320,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":30851072,"data_stored":61702144,"data_compressed":3185142,"data_compressed_allocated":30851072,"data_compressed_original":61702144,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":23666688,"data_stored":47333376,"data_compressed":2443366,"data_compressed_allocated":23666688,"data_compressed_original":47333376,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":30392320,"data_stored":60784640,"data_compressed":3137798,"data_compressed_allocated":30392320,"data_compressed_original":60784640,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:48:07.869 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:48:08.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:08.002+0000 7fb769d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:08.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:08.009+0000 7fb769d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:08.386 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 63 16 9437 9421 1.16818 1.07812 0.107403 0.106878 2024-04-23T01:48:09.386 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 64 16 9581 9565 1.1675 1.125 0.204681 0.106956 2024-04-23T01:48:10.304 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:48:10.386 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 65 16 9729 9713 1.16733 1.15625 0.175003 0.10696 2024-04-23T01:48:10.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:10.447+0000 7f41cf99d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:10.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:10.455+0000 7f41cf99d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:11.386 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 66 16 9883 9867 1.16787 1.20312 0.0880634 0.106912 2024-04-23T01:48:12.386 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 67 16 10026 10010 1.16711 1.11719 0.0787339 0.106965 2024-04-23T01:48:12.644 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:48:12.785 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:48:12.785 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:48:12.786 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:48:12.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:12.787+0000 7fe480b9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:12.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:12.794+0000 7fe480b9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:12.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:12.926+0000 7f16a6f5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:12.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:12.938+0000 7f16a6f5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:13.290 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:48:13.294 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:48:13.311 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2024-04-23T01:48:11.583650+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":75866656,"num_objects":1152,"num_object_clones":0,"num_object_copies":3456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":9326,"num_write_kb":75406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9270,"ondisk_log_size":9270,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":318740,"kb_used_data":110132,"kb_used_omap":0,"kb_used_meta":208576,"kb_avail":377168620,"statfs":{"total":386547056640,"available":386220666880,"internally_reserved":0,"allocated":112775168,"data_stored":225311424,"data_compressed":11612433,"data_compressed_allocated":111730688,"data_compressed_original":224935936,"omap_allocated":0,"internal_metadata":213581824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11640832,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1421,"num_write_kb":11368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002267"},"pg_stats":[{"pgid":"4.f","version":"29'575","reported_seq":588,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:05.393774+0000","last_change":"2024-04-23T01:47:04.491534+0000","last_active":"2024-04-23T01:48:05.393774+0000","last_peered":"2024-04-23T01:48:05.393774+0000","last_clean":"2024-04-23T01:48:05.393774+0000","last_became_active":"2024-04-23T01:47:04.486910+0000","last_became_peered":"2024-04-23T01:47:04.486910+0000","last_unstale":"2024-04-23T01:48:05.393774+0000","last_undegraded":"2024-04-23T01:48:05.393774+0000","last_fullsized":"2024-04-23T01:48:05.393774+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718592,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":576,"num_write_kb":4608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"29'511","reported_seq":524,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:04.435648+0000","last_change":"2024-04-23T01:47:04.452063+0000","last_active":"2024-04-23T01:48:04.435648+0000","last_peered":"2024-04-23T01:48:04.435648+0000","last_clean":"2024-04-23T01:48:04.435648+0000","last_became_active":"2024-04-23T01:47:04.443666+0000","last_became_peered":"2024-04-23T01:47:04.443666+0000","last_unstale":"2024-04-23T01:48:04.435648+0000","last_undegraded":"2024-04-23T01:48:04.435648+0000","last_fullsized":"2024-04-23T01:48:04.435648+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":512,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"29'543","reported_seq":556,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:05.465106+0000","last_change":"2024-04-23T01:47:04.485140+0000","last_active":"2024-04-23T01:48:05.465106+0000","last_peered":"2024-04-23T01:48:05.465106+0000","last_clean":"2024-04-23T01:48:05.465106+0000","last_became_active":"2024-04-23T01:47:04.480664+0000","last_became_peered":"2024-04-23T01:47:04.480664+0000","last_unstale":"2024-04-23T01:48:05.465106+0000","last_undegraded":"2024-04-23T01:48:05.465106+0000","last_fullsized":"2024-04-23T01:48:05.465106+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":544,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"29'559","reported_seq":572,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:05.771594+0000","last_change":"2024-04-23T01:47:04.442089+0000","last_active":"2024-04-23T01:48:05.771594+0000","last_peered":"2024-04-23T01:48:05.771594+0000","last_clean":"2024-04-23T01:48:05.771594+0000","last_became_active":"2024-04-23T01:47:04.431317+0000","last_became_peered":"2024-04-23T01:47:04.431317+0000","last_unstale":"2024-04-23T01:48:05.771594+0000","last_undegraded":"2024-04-23T01:48:05.771594+0000","last_fullsized":"2024-04-23T01:48:05.771594+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4587520,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":560,"num_write_kb":4480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"29'559","reported_seq":572,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:05.491475+0000","last_change":"2024-04-23T01:47:04.503686+0000","last_active":"2024-04-23T01:48:05.491475+0000","last_peered":"2024-04-23T01:48:05.491475+0000","last_clean":"2024-04-23T01:48:05.491475+0000","last_became_active":"2024-04-23T01:47:04.499155+0000","last_became_peered":"2024-04-23T01:47:04.499155+0000","last_unstale":"2024-04-23T01:48:05.491475+0000","last_undegraded":"2024-04-23T01:48:05.491475+0000","last_fullsized":"2024-04-23T01:48:05.491475+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4587520,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":560,"num_write_kb":4480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"29'663","reported_seq":676,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:06.275417+0000","last_change":"2024-04-23T01:47:04.516672+0000","last_active":"2024-04-23T01:48:06.275417+0000","last_peered":"2024-04-23T01:48:06.275417+0000","last_clean":"2024-04-23T01:48:06.275417+0000","last_became_active":"2024-04-23T01:47:04.504711+0000","last_became_peered":"2024-04-23T01:47:04.504711+0000","last_unstale":"2024-04-23T01:48:06.275417+0000","last_undegraded":"2024-04-23T01:48:06.275417+0000","last_fullsized":"2024-04-23T01:48:06.275417+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5439488,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":664,"num_write_kb":5312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"29'527","reported_seq":540,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:06.174653+0000","last_change":"2024-04-23T01:47:04.449451+0000","last_active":"2024-04-23T01:48:06.174653+0000","last_peered":"2024-04-23T01:48:06.174653+0000","last_clean":"2024-04-23T01:48:06.174653+0000","last_became_active":"2024-04-23T01:47:04.442623+0000","last_became_peered":"2024-04-23T01:47:04.442623+0000","last_unstale":"2024-04-23T01:48:06.174653+0000","last_undegraded":"2024-04-23T01:48:06.174653+0000","last_fullsized":"2024-04-23T01:48:06.174653+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":528,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"29'599","reported_seq":612,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:06.365653+0000","last_change":"2024-04-23T01:47:04.520179+0000","last_active":"2024-04-23T01:48:06.365653+0000","last_peered":"2024-04-23T01:48:06.365653+0000","last_clean":"2024-04-23T01:48:06.365653+0000","last_became_active":"2024-04-23T01:47:04.511239+0000","last_became_peered":"2024-04-23T01:47:04.511239+0000","last_unstale":"2024-04-23T01:48:06.365653+0000","last_undegraded":"2024-04-23T01:48:06.365653+0000","last_fullsized":"2024-04-23T01:48:06.365653+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"29'618","reported_seq":631,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:06.584808+0000","last_change":"2024-04-23T01:47:04.496716+0000","last_active":"2024-04-23T01:48:06.584808+0000","last_peered":"2024-04-23T01:48:06.584808+0000","last_clean":"2024-04-23T01:48:06.584808+0000","last_became_active":"2024-04-23T01:47:04.486607+0000","last_became_peered":"2024-04-23T01:47:04.486607+0000","last_unstale":"2024-04-23T01:48:06.584808+0000","last_undegraded":"2024-04-23T01:48:06.584808+0000","last_fullsized":"2024-04-23T01:48:06.584808+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5070848,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":619,"num_write_kb":4952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":186,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.325282+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:47:05.325282+0000","last_peered":"2024-04-23T01:47:05.325282+0000","last_clean":"2024-04-23T01:47:05.325282+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:47:05.325282+0000","last_undegraded":"2024-04-23T01:47:05.325282+0000","last_fullsized":"2024-04-23T01:47:05.325282+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.410555+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:47:05.410555+0000","last_peered":"2024-04-23T01:47:05.410555+0000","last_clean":"2024-04-23T01:47:05.410555+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:47:05.410555+0000","last_undegraded":"2024-04-23T01:47:05.410555+0000","last_fullsized":"2024-04-23T01:47:05.410555+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"29'519","reported_seq":532,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:06.218807+0000","last_change":"2024-04-23T01:47:04.467386+0000","last_active":"2024-04-23T01:48:06.218807+0000","last_peered":"2024-04-23T01:48:06.218807+0000","last_clean":"2024-04-23T01:48:06.218807+0000","last_became_active":"2024-04-23T01:47:04.459042+0000","last_became_peered":"2024-04-23T01:47:04.459042+0000","last_unstale":"2024-04-23T01:48:06.218807+0000","last_undegraded":"2024-04-23T01:48:06.218807+0000","last_fullsized":"2024-04-23T01:48:06.218807+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259840,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":520,"num_write_kb":4160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.570797+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:47:05.570797+0000","last_peered":"2024-04-23T01:47:05.570797+0000","last_clean":"2024-04-23T01:47:05.570797+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:47:05.570797+0000","last_undegraded":"2024-04-23T01:47:05.570797+0000","last_fullsized":"2024-04-23T01:47:05.570797+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"29'562","reported_seq":575,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:06.564389+0000","last_change":"2024-04-23T01:47:04.466464+0000","last_active":"2024-04-23T01:48:06.564389+0000","last_peered":"2024-04-23T01:48:06.564389+0000","last_clean":"2024-04-23T01:48:06.564389+0000","last_became_active":"2024-04-23T01:47:04.460338+0000","last_became_peered":"2024-04-23T01:47:04.460338+0000","last_unstale":"2024-04-23T01:48:06.564389+0000","last_undegraded":"2024-04-23T01:48:06.564389+0000","last_fullsized":"2024-04-23T01:48:06.564389+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4612096,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":563,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.397538+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:47:05.397538+0000","last_peered":"2024-04-23T01:47:05.397538+0000","last_clean":"2024-04-23T01:47:05.397538+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:47:05.397538+0000","last_undegraded":"2024-04-23T01:47:05.397538+0000","last_fullsized":"2024-04-23T01:47:05.397538+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"29'607","reported_seq":620,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:05.627990+0000","last_change":"2024-04-23T01:47:04.435560+0000","last_active":"2024-04-23T01:48:05.627990+0000","last_peered":"2024-04-23T01:48:05.627990+0000","last_clean":"2024-04-23T01:48:05.627990+0000","last_became_active":"2024-04-23T01:47:04.424493+0000","last_became_peered":"2024-04-23T01:47:04.424493+0000","last_unstale":"2024-04-23T01:48:05.627990+0000","last_undegraded":"2024-04-23T01:48:05.627990+0000","last_fullsized":"2024-04-23T01:48:05.627990+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4980736,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":608,"num_write_kb":4864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.576933+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:47:05.576933+0000","last_peered":"2024-04-23T01:47:05.576933+0000","last_clean":"2024-04-23T01:47:05.576933+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:47:05.576933+0000","last_undegraded":"2024-04-23T01:47:05.576933+0000","last_fullsized":"2024-04-23T01:47:05.576933+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"29'567","reported_seq":580,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:06.466436+0000","last_change":"2024-04-23T01:47:04.530152+0000","last_active":"2024-04-23T01:48:06.466436+0000","last_peered":"2024-04-23T01:48:06.466436+0000","last_clean":"2024-04-23T01:48:06.466436+0000","last_became_active":"2024-04-23T01:47:04.519844+0000","last_became_peered":"2024-04-23T01:47:04.519844+0000","last_unstale":"2024-04-23T01:48:06.466436+0000","last_undegraded":"2024-04-23T01:48:06.466436+0000","last_fullsized":"2024-04-23T01:48:06.466436+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":568,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.590971+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:47:05.590971+0000","last_peered":"2024-04-23T01:47:05.590971+0000","last_clean":"2024-04-23T01:47:05.590971+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:47:05.590971+0000","last_undegraded":"2024-04-23T01:47:05.590971+0000","last_fullsized":"2024-04-23T01:47:05.590971+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"29'662","reported_seq":675,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:06.549318+0000","last_change":"2024-04-23T01:47:04.465832+0000","last_active":"2024-04-23T01:48:06.549318+0000","last_peered":"2024-04-23T01:48:06.549318+0000","last_clean":"2024-04-23T01:48:06.549318+0000","last_became_active":"2024-04-23T01:47:04.460695+0000","last_became_peered":"2024-04-23T01:47:04.460695+0000","last_unstale":"2024-04-23T01:48:06.549318+0000","last_undegraded":"2024-04-23T01:48:06.549318+0000","last_fullsized":"2024-04-23T01:48:06.549318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5431296,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":663,"num_write_kb":5304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.331332+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:47:05.331332+0000","last_peered":"2024-04-23T01:47:05.331332+0000","last_clean":"2024-04-23T01:47:05.331332+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:47:05.331332+0000","last_undegraded":"2024-04-23T01:47:05.331332+0000","last_fullsized":"2024-04-23T01:47:05.331332+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"29'591","reported_seq":604,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:05.185395+0000","last_change":"2024-04-23T01:47:04.434787+0000","last_active":"2024-04-23T01:48:05.185395+0000","last_peered":"2024-04-23T01:48:05.185395+0000","last_clean":"2024-04-23T01:48:05.185395+0000","last_became_active":"2024-04-23T01:47:04.423352+0000","last_became_peered":"2024-04-23T01:47:04.423352+0000","last_unstale":"2024-04-23T01:48:05.185395+0000","last_undegraded":"2024-04-23T01:48:05.185395+0000","last_fullsized":"2024-04-23T01:48:05.185395+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":592,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.602643+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:47:05.602643+0000","last_peered":"2024-04-23T01:47:05.602643+0000","last_clean":"2024-04-23T01:47:05.602643+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:47:05.602643+0000","last_undegraded":"2024-04-23T01:47:05.602643+0000","last_fullsized":"2024-04-23T01:47:05.602643+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"29'511","reported_seq":524,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:02.068547+0000","last_change":"2024-04-23T01:47:04.425726+0000","last_active":"2024-04-23T01:48:02.068547+0000","last_peered":"2024-04-23T01:48:02.068547+0000","last_clean":"2024-04-23T01:48:02.068547+0000","last_became_active":"2024-04-23T01:47:04.416905+0000","last_became_peered":"2024-04-23T01:47:04.416905+0000","last_unstale":"2024-04-23T01:48:02.068547+0000","last_undegraded":"2024-04-23T01:48:02.068547+0000","last_fullsized":"2024-04-23T01:48:02.068547+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":512,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.400601+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:47:05.400601+0000","last_peered":"2024-04-23T01:47:05.400601+0000","last_clean":"2024-04-23T01:47:05.400601+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:47:05.400601+0000","last_undegraded":"2024-04-23T01:47:05.400601+0000","last_fullsized":"2024-04-23T01:47:05.400601+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":75276288,"num_objects":1150,"num_object_clones":0,"num_object_copies":3450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9189,"num_write_kb":73512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":112906240,"data_stored":225812480,"data_compressed":11656665,"data_compressed_allocated":112906240,"data_compressed_original":225812480,"omap_allocated":0,"internal_metadata":0},"log_size":9173,"ondisk_log_size":9173,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705843,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":74524,"kb_used_data":29652,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94297316,"statfs":{"total":96636764160,"available":96560451584,"internally_reserved":0,"allocated":30363648,"data_stored":60784440,"data_compressed":3133065,"data_compressed_allocated":30097408,"data_compressed_original":60686336,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705843,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":84656,"kb_used_data":23144,"kb_used_omap":0,"kb_used_meta":61504,"kb_avail":94287184,"statfs":{"total":96636764160,"available":96550076416,"internally_reserved":0,"allocated":23699456,"data_stored":47456056,"data_compressed":2444978,"data_compressed_allocated":23433216,"data_compressed_original":47357952,"omap_allocated":0,"internal_metadata":62980096},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705843,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":83176,"kb_used_data":29984,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94288664,"statfs":{"total":96636764160,"available":96551591936,"internally_reserved":0,"allocated":30703616,"data_stored":60996888,"data_compressed":3144534,"data_compressed_allocated":30457856,"data_compressed_original":60915712,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705843,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":76384,"kb_used_data":27352,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94295456,"statfs":{"total":96636764160,"available":96558546944,"internally_reserved":0,"allocated":28008448,"data_stored":56074040,"data_compressed":2889856,"data_compressed_allocated":27742208,"data_compressed_original":55975936,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":27996160,"data_stored":55992320,"data_compressed":2890359,"data_compressed_allocated":27996160,"data_compressed_original":55992320,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":30851072,"data_stored":61702144,"data_compressed":3185142,"data_compressed_allocated":30851072,"data_compressed_original":61702144,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":23666688,"data_stored":47333376,"data_compressed":2443366,"data_compressed_allocated":23666688,"data_compressed_original":47333376,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":30392320,"data_stored":60784640,"data_compressed":3137798,"data_compressed_allocated":30392320,"data_compressed_original":60784640,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:48:13.386 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 68 16 10152 10136 1.16442 0.984375 0.0805033 0.107221 2024-04-23T01:48:14.386 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 69 16 10288 10272 1.16294 1.0625 0.213423 0.107341 2024-04-23T01:48:15.003 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:48:15.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:15.131+0000 7fab10a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:15.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:15.138+0000 7fab10a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:15.386 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 70 16 10433 10417 1.16251 1.13281 0.0957393 0.107419 2024-04-23T01:48:16.386 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 71 16 10582 10566 1.16253 1.16406 0.229569 0.107398 2024-04-23T01:48:17.323 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:48:17.387 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 72 16 10719 10703 1.16125 1.07031 0.0237655 0.107541 2024-04-23T01:48:17.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:17.457+0000 7f566eeed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:17.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:17.464+0000 7f566eeed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:18.313 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:48:18.313 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:48:18.314 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:48:18.387 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 73 16 10869 10853 1.16139 1.17188 0.106098 0.107522 2024-04-23T01:48:18.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:18.438+0000 7f4879154640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:18.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:18.447+0000 7f4879154640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:18.790 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:48:18.791 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:48:18.809 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2024-04-23T01:48:17.584997+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":87384608,"num_objects":1327,"num_object_clones":0,"num_object_copies":3981,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":10732,"num_write_kb":86654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10676,"ondisk_log_size":10676,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":331380,"kb_used_data":126932,"kb_used_omap":0,"kb_used_meta":204416,"kb_avail":377155980,"statfs":{"total":386547056640,"available":386207723520,"internally_reserved":0,"allocated":129978368,"data_stored":259717824,"data_compressed":13389033,"data_compressed_allocated":128933888,"data_compressed_original":259342336,"omap_allocated":0,"internal_metadata":209321984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17383424,"num_objects":265,"num_object_clones":0,"num_object_copies":795,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2122,"num_write_kb":16976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002457"},"pg_stats":[{"pgid":"4.f","version":"29'671","reported_seq":684,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:15.290145+0000","last_change":"2024-04-23T01:47:04.491534+0000","last_active":"2024-04-23T01:48:15.290145+0000","last_peered":"2024-04-23T01:48:15.290145+0000","last_clean":"2024-04-23T01:48:15.290145+0000","last_became_active":"2024-04-23T01:47:04.486910+0000","last_became_peered":"2024-04-23T01:47:04.486910+0000","last_unstale":"2024-04-23T01:48:15.290145+0000","last_undegraded":"2024-04-23T01:48:15.290145+0000","last_fullsized":"2024-04-23T01:48:15.290145+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505024,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":672,"num_write_kb":5376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"29'631","reported_seq":644,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:16.055470+0000","last_change":"2024-04-23T01:47:04.452063+0000","last_active":"2024-04-23T01:48:16.055470+0000","last_peered":"2024-04-23T01:48:16.055470+0000","last_clean":"2024-04-23T01:48:16.055470+0000","last_became_active":"2024-04-23T01:47:04.443666+0000","last_became_peered":"2024-04-23T01:47:04.443666+0000","last_unstale":"2024-04-23T01:48:16.055470+0000","last_undegraded":"2024-04-23T01:48:16.055470+0000","last_fullsized":"2024-04-23T01:48:16.055470+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":5056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"29'647","reported_seq":660,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:16.588443+0000","last_change":"2024-04-23T01:47:04.485140+0000","last_active":"2024-04-23T01:48:16.588443+0000","last_peered":"2024-04-23T01:48:16.588443+0000","last_clean":"2024-04-23T01:48:16.588443+0000","last_became_active":"2024-04-23T01:47:04.480664+0000","last_became_peered":"2024-04-23T01:47:04.480664+0000","last_unstale":"2024-04-23T01:48:16.588443+0000","last_undegraded":"2024-04-23T01:48:16.588443+0000","last_fullsized":"2024-04-23T01:48:16.588443+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308416,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":648,"num_write_kb":5184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"29'703","reported_seq":716,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:16.360365+0000","last_change":"2024-04-23T01:47:04.442089+0000","last_active":"2024-04-23T01:48:16.360365+0000","last_peered":"2024-04-23T01:48:16.360365+0000","last_clean":"2024-04-23T01:48:16.360365+0000","last_became_active":"2024-04-23T01:47:04.431317+0000","last_became_peered":"2024-04-23T01:47:04.431317+0000","last_unstale":"2024-04-23T01:48:16.360365+0000","last_undegraded":"2024-04-23T01:48:16.360365+0000","last_fullsized":"2024-04-23T01:48:16.360365+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767168,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":704,"num_write_kb":5632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"29'647","reported_seq":660,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:14.947716+0000","last_change":"2024-04-23T01:47:04.503686+0000","last_active":"2024-04-23T01:48:14.947716+0000","last_peered":"2024-04-23T01:48:14.947716+0000","last_clean":"2024-04-23T01:48:14.947716+0000","last_became_active":"2024-04-23T01:47:04.499155+0000","last_became_peered":"2024-04-23T01:47:04.499155+0000","last_unstale":"2024-04-23T01:48:14.947716+0000","last_undegraded":"2024-04-23T01:48:14.947716+0000","last_fullsized":"2024-04-23T01:48:14.947716+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308416,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":648,"num_write_kb":5184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"29'751","reported_seq":764,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:16.565048+0000","last_change":"2024-04-23T01:47:04.516672+0000","last_active":"2024-04-23T01:48:16.565048+0000","last_peered":"2024-04-23T01:48:16.565048+0000","last_clean":"2024-04-23T01:48:16.565048+0000","last_became_active":"2024-04-23T01:47:04.504711+0000","last_became_peered":"2024-04-23T01:47:04.504711+0000","last_unstale":"2024-04-23T01:48:16.565048+0000","last_undegraded":"2024-04-23T01:48:16.565048+0000","last_fullsized":"2024-04-23T01:48:16.565048+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"29'567","reported_seq":580,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:15.511661+0000","last_change":"2024-04-23T01:47:04.449451+0000","last_active":"2024-04-23T01:48:15.511661+0000","last_peered":"2024-04-23T01:48:15.511661+0000","last_clean":"2024-04-23T01:48:15.511661+0000","last_became_active":"2024-04-23T01:47:04.442623+0000","last_became_peered":"2024-04-23T01:47:04.442623+0000","last_unstale":"2024-04-23T01:48:15.511661+0000","last_undegraded":"2024-04-23T01:48:15.511661+0000","last_fullsized":"2024-04-23T01:48:15.511661+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":568,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"29'703","reported_seq":716,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:15.627386+0000","last_change":"2024-04-23T01:47:04.520179+0000","last_active":"2024-04-23T01:48:15.627386+0000","last_peered":"2024-04-23T01:48:15.627386+0000","last_clean":"2024-04-23T01:48:15.627386+0000","last_became_active":"2024-04-23T01:47:04.511239+0000","last_became_peered":"2024-04-23T01:47:04.511239+0000","last_unstale":"2024-04-23T01:48:15.627386+0000","last_undegraded":"2024-04-23T01:48:15.627386+0000","last_fullsized":"2024-04-23T01:48:15.627386+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767168,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":704,"num_write_kb":5632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"29'711","reported_seq":724,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:16.464790+0000","last_change":"2024-04-23T01:47:04.496716+0000","last_active":"2024-04-23T01:48:16.464790+0000","last_peered":"2024-04-23T01:48:16.464790+0000","last_clean":"2024-04-23T01:48:16.464790+0000","last_became_active":"2024-04-23T01:47:04.486607+0000","last_became_peered":"2024-04-23T01:47:04.486607+0000","last_unstale":"2024-04-23T01:48:16.464790+0000","last_undegraded":"2024-04-23T01:48:16.464790+0000","last_fullsized":"2024-04-23T01:48:16.464790+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5832704,"num_objects":89,"num_object_clones":0,"num_object_copies":267,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":712,"num_write_kb":5696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":186,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.325282+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:47:05.325282+0000","last_peered":"2024-04-23T01:47:05.325282+0000","last_clean":"2024-04-23T01:47:05.325282+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:47:05.325282+0000","last_undegraded":"2024-04-23T01:47:05.325282+0000","last_fullsized":"2024-04-23T01:47:05.325282+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.410555+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:47:05.410555+0000","last_peered":"2024-04-23T01:47:05.410555+0000","last_clean":"2024-04-23T01:47:05.410555+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:47:05.410555+0000","last_undegraded":"2024-04-23T01:47:05.410555+0000","last_fullsized":"2024-04-23T01:47:05.410555+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"29'655","reported_seq":668,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:16.174351+0000","last_change":"2024-04-23T01:47:04.467386+0000","last_active":"2024-04-23T01:48:16.174351+0000","last_peered":"2024-04-23T01:48:16.174351+0000","last_clean":"2024-04-23T01:48:16.174351+0000","last_became_active":"2024-04-23T01:47:04.459042+0000","last_became_peered":"2024-04-23T01:47:04.459042+0000","last_unstale":"2024-04-23T01:48:16.174351+0000","last_undegraded":"2024-04-23T01:48:16.174351+0000","last_fullsized":"2024-04-23T01:48:16.174351+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.570797+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:47:05.570797+0000","last_peered":"2024-04-23T01:47:05.570797+0000","last_clean":"2024-04-23T01:47:05.570797+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:47:05.570797+0000","last_undegraded":"2024-04-23T01:47:05.570797+0000","last_fullsized":"2024-04-23T01:47:05.570797+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"29'615","reported_seq":628,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:15.739817+0000","last_change":"2024-04-23T01:47:04.466464+0000","last_active":"2024-04-23T01:48:15.739817+0000","last_peered":"2024-04-23T01:48:15.739817+0000","last_clean":"2024-04-23T01:48:15.739817+0000","last_became_active":"2024-04-23T01:47:04.460338+0000","last_became_peered":"2024-04-23T01:47:04.460338+0000","last_unstale":"2024-04-23T01:48:15.739817+0000","last_undegraded":"2024-04-23T01:48:15.739817+0000","last_fullsized":"2024-04-23T01:48:15.739817+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5046272,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":616,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.397538+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:47:05.397538+0000","last_peered":"2024-04-23T01:47:05.397538+0000","last_clean":"2024-04-23T01:47:05.397538+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:47:05.397538+0000","last_undegraded":"2024-04-23T01:47:05.397538+0000","last_fullsized":"2024-04-23T01:47:05.397538+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"29'647","reported_seq":660,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:15.337336+0000","last_change":"2024-04-23T01:47:04.435560+0000","last_active":"2024-04-23T01:48:15.337336+0000","last_peered":"2024-04-23T01:48:15.337336+0000","last_clean":"2024-04-23T01:48:15.337336+0000","last_became_active":"2024-04-23T01:47:04.424493+0000","last_became_peered":"2024-04-23T01:47:04.424493+0000","last_unstale":"2024-04-23T01:48:15.337336+0000","last_undegraded":"2024-04-23T01:48:15.337336+0000","last_fullsized":"2024-04-23T01:48:15.337336+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308416,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":648,"num_write_kb":5184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.576933+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:47:05.576933+0000","last_peered":"2024-04-23T01:47:05.576933+0000","last_clean":"2024-04-23T01:47:05.576933+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:47:05.576933+0000","last_undegraded":"2024-04-23T01:47:05.576933+0000","last_fullsized":"2024-04-23T01:47:05.576933+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"29'602","reported_seq":615,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:16.576326+0000","last_change":"2024-04-23T01:47:04.530152+0000","last_active":"2024-04-23T01:48:16.576326+0000","last_peered":"2024-04-23T01:48:16.576326+0000","last_clean":"2024-04-23T01:48:16.576326+0000","last_became_active":"2024-04-23T01:47:04.519844+0000","last_became_peered":"2024-04-23T01:47:04.519844+0000","last_unstale":"2024-04-23T01:48:16.576326+0000","last_undegraded":"2024-04-23T01:48:16.576326+0000","last_fullsized":"2024-04-23T01:48:16.576326+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4939776,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":603,"num_write_kb":4824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.590971+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:47:05.590971+0000","last_peered":"2024-04-23T01:47:05.590971+0000","last_clean":"2024-04-23T01:47:05.590971+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:47:05.590971+0000","last_undegraded":"2024-04-23T01:47:05.590971+0000","last_fullsized":"2024-04-23T01:47:05.590971+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"29'759","reported_seq":772,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:16.402370+0000","last_change":"2024-04-23T01:47:04.465832+0000","last_active":"2024-04-23T01:48:16.402370+0000","last_peered":"2024-04-23T01:48:16.402370+0000","last_clean":"2024-04-23T01:48:16.402370+0000","last_became_active":"2024-04-23T01:47:04.460695+0000","last_became_peered":"2024-04-23T01:47:04.460695+0000","last_unstale":"2024-04-23T01:48:16.402370+0000","last_undegraded":"2024-04-23T01:48:16.402370+0000","last_fullsized":"2024-04-23T01:48:16.402370+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6225920,"num_objects":95,"num_object_clones":0,"num_object_copies":285,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":760,"num_write_kb":6080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.331332+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:47:05.331332+0000","last_peered":"2024-04-23T01:47:05.331332+0000","last_clean":"2024-04-23T01:47:05.331332+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:47:05.331332+0000","last_undegraded":"2024-04-23T01:47:05.331332+0000","last_fullsized":"2024-04-23T01:47:05.331332+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"29'695","reported_seq":708,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:16.054208+0000","last_change":"2024-04-23T01:47:04.434787+0000","last_active":"2024-04-23T01:48:16.054208+0000","last_peered":"2024-04-23T01:48:16.054208+0000","last_clean":"2024-04-23T01:48:16.054208+0000","last_became_active":"2024-04-23T01:47:04.423352+0000","last_became_peered":"2024-04-23T01:47:04.423352+0000","last_unstale":"2024-04-23T01:48:16.054208+0000","last_undegraded":"2024-04-23T01:48:16.054208+0000","last_fullsized":"2024-04-23T01:48:16.054208+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":696,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.602643+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:47:05.602643+0000","last_peered":"2024-04-23T01:47:05.602643+0000","last_clean":"2024-04-23T01:47:05.602643+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:47:05.602643+0000","last_undegraded":"2024-04-23T01:47:05.602643+0000","last_fullsized":"2024-04-23T01:47:05.602643+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"29'575","reported_seq":588,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:15.367323+0000","last_change":"2024-04-23T01:47:04.425726+0000","last_active":"2024-04-23T01:48:15.367323+0000","last_peered":"2024-04-23T01:48:15.367323+0000","last_clean":"2024-04-23T01:48:15.367323+0000","last_became_active":"2024-04-23T01:47:04.416905+0000","last_became_peered":"2024-04-23T01:47:04.416905+0000","last_unstale":"2024-04-23T01:48:15.367323+0000","last_undegraded":"2024-04-23T01:48:15.367323+0000","last_fullsized":"2024-04-23T01:48:15.367323+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718592,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":576,"num_write_kb":4608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.400601+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:47:05.400601+0000","last_peered":"2024-04-23T01:47:05.400601+0000","last_clean":"2024-04-23T01:47:05.400601+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:47:05.400601+0000","last_undegraded":"2024-04-23T01:47:05.400601+0000","last_fullsized":"2024-04-23T01:47:05.400601+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":86794240,"num_objects":1325,"num_object_clones":0,"num_object_copies":3975,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10595,"num_write_kb":84760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":130170880,"data_stored":260341760,"data_compressed":13439610,"data_compressed_allocated":130170880,"data_compressed_original":260341760,"omap_allocated":0,"internal_metadata":0},"log_size":10579,"ondisk_log_size":10579,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":83168,"kb_used_data":34136,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94288672,"statfs":{"total":96636764160,"available":96551600128,"internally_reserved":0,"allocated":34955264,"data_stored":69967672,"data_compressed":3607248,"data_compressed_allocated":34689024,"data_compressed_original":69869568,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":71440,"kb_used_data":26568,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94300400,"statfs":{"total":96636764160,"available":96563609600,"internally_reserved":0,"allocated":27205632,"data_stored":54468408,"data_compressed":2807066,"data_compressed_allocated":26939392,"data_compressed_original":54370304,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":91764,"kb_used_data":34412,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94280076,"statfs":{"total":96636764160,"available":96542797824,"internally_reserved":0,"allocated":35237888,"data_stored":70065432,"data_compressed":3612795,"data_compressed_allocated":34992128,"data_compressed_original":69984256,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":85008,"kb_used_data":31816,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94286832,"statfs":{"total":96636764160,"available":96549715968,"internally_reserved":0,"allocated":32579584,"data_stored":65216312,"data_compressed":3361924,"data_compressed_allocated":32313344,"data_compressed_original":65118208,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32595968,"data_stored":65191936,"data_compressed":3365388,"data_compressed_allocated":32595968,"data_compressed_original":65191936,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":35438592,"data_stored":70877184,"data_compressed":3658902,"data_compressed_allocated":35438592,"data_compressed_original":70877184,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27168768,"data_stored":54337536,"data_compressed":2805031,"data_compressed_allocated":27168768,"data_compressed_original":54337536,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":34967552,"data_stored":69935104,"data_compressed":3610289,"data_compressed_allocated":34967552,"data_compressed_original":69935104,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:48:19.387 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 74 16 11010 10994 1.16058 1.10156 0.150365 0.107617 2024-04-23T01:48:19.632 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:48:19.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:19.748+0000 7fd3ee99d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:19.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:19.755+0000 7fd3ee99d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:20.387 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 75 16 11138 11122 1.15844 1 0.0847858 0.107748 2024-04-23T01:48:21.387 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 76 16 11278 11262 1.15759 1.09375 0.128298 0.107892 2024-04-23T01:48:22.047 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:48:22.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:22.160+0000 7fef1791d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:22.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:22.167+0000 7fef1791d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:22.387 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 77 16 11408 11392 1.15574 1.01562 0.247769 0.108048 2024-04-23T01:48:23.387 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 78 16 11547 11531 1.15485 1.08594 0.16547 0.108102 2024-04-23T01:48:23.812 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:48:23.812 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:48:23.812 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:48:23.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:23.935+0000 7fa0b099d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:23.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:23.944+0000 7fa0b099d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:24.315 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:48:24.316 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:48:24.330 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2024-04-23T01:48:23.586120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":93143584,"num_objects":1415,"num_object_clones":0,"num_object_copies":4245,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":11435,"num_write_kb":92278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11379,"ondisk_log_size":11379,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":331372,"kb_used_data":135240,"kb_used_omap":0,"kb_used_meta":196096,"kb_avail":377155988,"statfs":{"total":386547056640,"available":386207731712,"internally_reserved":0,"allocated":138485760,"data_stored":276732608,"data_compressed":14267601,"data_compressed_allocated":137441280,"data_compressed_original":276357120,"omap_allocated":0,"internal_metadata":200802304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17276928,"num_objects":263,"num_object_clones":0,"num_object_copies":789,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2109,"num_write_kb":16872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002470"},"pg_stats":[{"pgid":"4.f","version":"29'703","reported_seq":716,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:21.316729+0000","last_change":"2024-04-23T01:47:04.491534+0000","last_active":"2024-04-23T01:48:21.316729+0000","last_peered":"2024-04-23T01:48:21.316729+0000","last_clean":"2024-04-23T01:48:21.316729+0000","last_became_active":"2024-04-23T01:47:04.486910+0000","last_became_peered":"2024-04-23T01:47:04.486910+0000","last_unstale":"2024-04-23T01:48:21.316729+0000","last_undegraded":"2024-04-23T01:48:21.316729+0000","last_fullsized":"2024-04-23T01:48:21.316729+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767168,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":704,"num_write_kb":5632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"29'703","reported_seq":716,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:21.060362+0000","last_change":"2024-04-23T01:47:04.452063+0000","last_active":"2024-04-23T01:48:21.060362+0000","last_peered":"2024-04-23T01:48:21.060362+0000","last_clean":"2024-04-23T01:48:21.060362+0000","last_became_active":"2024-04-23T01:47:04.443666+0000","last_became_peered":"2024-04-23T01:47:04.443666+0000","last_unstale":"2024-04-23T01:48:21.060362+0000","last_undegraded":"2024-04-23T01:48:21.060362+0000","last_fullsized":"2024-04-23T01:48:21.060362+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767168,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":704,"num_write_kb":5632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"29'710","reported_seq":723,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:21.574524+0000","last_change":"2024-04-23T01:47:04.485140+0000","last_active":"2024-04-23T01:48:21.574524+0000","last_peered":"2024-04-23T01:48:21.574524+0000","last_clean":"2024-04-23T01:48:21.574524+0000","last_became_active":"2024-04-23T01:47:04.480664+0000","last_became_peered":"2024-04-23T01:47:04.480664+0000","last_unstale":"2024-04-23T01:48:21.574524+0000","last_undegraded":"2024-04-23T01:48:21.574524+0000","last_fullsized":"2024-04-23T01:48:21.574524+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5824512,"num_objects":89,"num_object_clones":0,"num_object_copies":267,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":711,"num_write_kb":5688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"29'735","reported_seq":748,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:21.203008+0000","last_change":"2024-04-23T01:47:04.442089+0000","last_active":"2024-04-23T01:48:21.203008+0000","last_peered":"2024-04-23T01:48:21.203008+0000","last_clean":"2024-04-23T01:48:21.203008+0000","last_became_active":"2024-04-23T01:47:04.431317+0000","last_became_peered":"2024-04-23T01:47:04.431317+0000","last_unstale":"2024-04-23T01:48:21.203008+0000","last_undegraded":"2024-04-23T01:48:21.203008+0000","last_fullsized":"2024-04-23T01:48:21.203008+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6029312,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":736,"num_write_kb":5888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"29'719","reported_seq":732,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:21.546027+0000","last_change":"2024-04-23T01:47:04.503686+0000","last_active":"2024-04-23T01:48:21.546027+0000","last_peered":"2024-04-23T01:48:21.546027+0000","last_clean":"2024-04-23T01:48:21.546027+0000","last_became_active":"2024-04-23T01:47:04.499155+0000","last_became_peered":"2024-04-23T01:47:04.499155+0000","last_unstale":"2024-04-23T01:48:21.546027+0000","last_undegraded":"2024-04-23T01:48:21.546027+0000","last_fullsized":"2024-04-23T01:48:21.546027+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":720,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"29'815","reported_seq":828,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:20.113580+0000","last_change":"2024-04-23T01:47:04.516672+0000","last_active":"2024-04-23T01:48:20.113580+0000","last_peered":"2024-04-23T01:48:20.113580+0000","last_clean":"2024-04-23T01:48:20.113580+0000","last_became_active":"2024-04-23T01:47:04.504711+0000","last_became_peered":"2024-04-23T01:47:04.504711+0000","last_unstale":"2024-04-23T01:48:20.113580+0000","last_undegraded":"2024-04-23T01:48:20.113580+0000","last_fullsized":"2024-04-23T01:48:20.113580+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6684672,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":816,"num_write_kb":6528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"29'599","reported_seq":612,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:21.254858+0000","last_change":"2024-04-23T01:47:04.449451+0000","last_active":"2024-04-23T01:48:21.254858+0000","last_peered":"2024-04-23T01:48:21.254858+0000","last_clean":"2024-04-23T01:48:21.254858+0000","last_became_active":"2024-04-23T01:47:04.442623+0000","last_became_peered":"2024-04-23T01:47:04.442623+0000","last_unstale":"2024-04-23T01:48:21.254858+0000","last_undegraded":"2024-04-23T01:48:21.254858+0000","last_fullsized":"2024-04-23T01:48:21.254858+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"29'727","reported_seq":740,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:21.464772+0000","last_change":"2024-04-23T01:47:04.520179+0000","last_active":"2024-04-23T01:48:21.464772+0000","last_peered":"2024-04-23T01:48:21.464772+0000","last_clean":"2024-04-23T01:48:21.464772+0000","last_became_active":"2024-04-23T01:47:04.511239+0000","last_became_peered":"2024-04-23T01:47:04.511239+0000","last_unstale":"2024-04-23T01:48:21.464772+0000","last_undegraded":"2024-04-23T01:48:21.464772+0000","last_fullsized":"2024-04-23T01:48:21.464772+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":728,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"29'743","reported_seq":756,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:19.597844+0000","last_change":"2024-04-23T01:47:04.496716+0000","last_active":"2024-04-23T01:48:19.597844+0000","last_peered":"2024-04-23T01:48:19.597844+0000","last_clean":"2024-04-23T01:48:19.597844+0000","last_became_active":"2024-04-23T01:47:04.486607+0000","last_became_peered":"2024-04-23T01:47:04.486607+0000","last_unstale":"2024-04-23T01:48:19.597844+0000","last_undegraded":"2024-04-23T01:48:19.597844+0000","last_fullsized":"2024-04-23T01:48:19.597844+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6094848,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":744,"num_write_kb":5952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":186,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.325282+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:47:05.325282+0000","last_peered":"2024-04-23T01:47:05.325282+0000","last_clean":"2024-04-23T01:47:05.325282+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:47:05.325282+0000","last_undegraded":"2024-04-23T01:47:05.325282+0000","last_fullsized":"2024-04-23T01:47:05.325282+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.410555+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:47:05.410555+0000","last_peered":"2024-04-23T01:47:05.410555+0000","last_clean":"2024-04-23T01:47:05.410555+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:47:05.410555+0000","last_undegraded":"2024-04-23T01:47:05.410555+0000","last_fullsized":"2024-04-23T01:47:05.410555+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"29'687","reported_seq":700,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:19.977431+0000","last_change":"2024-04-23T01:47:04.467386+0000","last_active":"2024-04-23T01:48:19.977431+0000","last_peered":"2024-04-23T01:48:19.977431+0000","last_clean":"2024-04-23T01:48:19.977431+0000","last_became_active":"2024-04-23T01:47:04.459042+0000","last_became_peered":"2024-04-23T01:47:04.459042+0000","last_unstale":"2024-04-23T01:48:19.977431+0000","last_undegraded":"2024-04-23T01:48:19.977431+0000","last_fullsized":"2024-04-23T01:48:19.977431+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5636096,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":688,"num_write_kb":5504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.570797+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:47:05.570797+0000","last_peered":"2024-04-23T01:47:05.570797+0000","last_clean":"2024-04-23T01:47:05.570797+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:47:05.570797+0000","last_undegraded":"2024-04-23T01:47:05.570797+0000","last_fullsized":"2024-04-23T01:47:05.570797+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"29'631","reported_seq":644,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:19.780363+0000","last_change":"2024-04-23T01:47:04.466464+0000","last_active":"2024-04-23T01:48:19.780363+0000","last_peered":"2024-04-23T01:48:19.780363+0000","last_clean":"2024-04-23T01:48:19.780363+0000","last_became_active":"2024-04-23T01:47:04.460338+0000","last_became_peered":"2024-04-23T01:47:04.460338+0000","last_unstale":"2024-04-23T01:48:19.780363+0000","last_undegraded":"2024-04-23T01:48:19.780363+0000","last_fullsized":"2024-04-23T01:48:19.780363+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":5056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.397538+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:47:05.397538+0000","last_peered":"2024-04-23T01:47:05.397538+0000","last_clean":"2024-04-23T01:47:05.397538+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:47:05.397538+0000","last_undegraded":"2024-04-23T01:47:05.397538+0000","last_fullsized":"2024-04-23T01:47:05.397538+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"29'671","reported_seq":684,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:21.375929+0000","last_change":"2024-04-23T01:47:04.435560+0000","last_active":"2024-04-23T01:48:21.375929+0000","last_peered":"2024-04-23T01:48:21.375929+0000","last_clean":"2024-04-23T01:48:21.375929+0000","last_became_active":"2024-04-23T01:47:04.424493+0000","last_became_peered":"2024-04-23T01:47:04.424493+0000","last_unstale":"2024-04-23T01:48:21.375929+0000","last_undegraded":"2024-04-23T01:48:21.375929+0000","last_fullsized":"2024-04-23T01:48:21.375929+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505024,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":672,"num_write_kb":5376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.576933+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:47:05.576933+0000","last_peered":"2024-04-23T01:47:05.576933+0000","last_clean":"2024-04-23T01:47:05.576933+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:47:05.576933+0000","last_undegraded":"2024-04-23T01:47:05.576933+0000","last_fullsized":"2024-04-23T01:47:05.576933+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"29'627","reported_seq":640,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:21.606799+0000","last_change":"2024-04-23T01:47:04.530152+0000","last_active":"2024-04-23T01:48:21.606799+0000","last_peered":"2024-04-23T01:48:21.606799+0000","last_clean":"2024-04-23T01:48:21.606799+0000","last_became_active":"2024-04-23T01:47:04.519844+0000","last_became_peered":"2024-04-23T01:47:04.519844+0000","last_unstale":"2024-04-23T01:48:21.606799+0000","last_undegraded":"2024-04-23T01:48:21.606799+0000","last_fullsized":"2024-04-23T01:48:21.606799+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5144576,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":628,"num_write_kb":5024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.590971+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:47:05.590971+0000","last_peered":"2024-04-23T01:47:05.590971+0000","last_clean":"2024-04-23T01:47:05.590971+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:47:05.590971+0000","last_undegraded":"2024-04-23T01:47:05.590971+0000","last_fullsized":"2024-04-23T01:47:05.590971+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"29'823","reported_seq":836,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:21.398434+0000","last_change":"2024-04-23T01:47:04.465832+0000","last_active":"2024-04-23T01:48:21.398434+0000","last_peered":"2024-04-23T01:48:21.398434+0000","last_clean":"2024-04-23T01:48:21.398434+0000","last_became_active":"2024-04-23T01:47:04.460695+0000","last_became_peered":"2024-04-23T01:47:04.460695+0000","last_unstale":"2024-04-23T01:48:21.398434+0000","last_undegraded":"2024-04-23T01:48:21.398434+0000","last_fullsized":"2024-04-23T01:48:21.398434+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":823,"log_dups_size":0,"ondisk_log_size":823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6750208,"num_objects":103,"num_object_clones":0,"num_object_copies":309,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":824,"num_write_kb":6592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.331332+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:47:05.331332+0000","last_peered":"2024-04-23T01:47:05.331332+0000","last_clean":"2024-04-23T01:47:05.331332+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:47:05.331332+0000","last_undegraded":"2024-04-23T01:47:05.331332+0000","last_fullsized":"2024-04-23T01:47:05.331332+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"29'775","reported_seq":788,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:20.861150+0000","last_change":"2024-04-23T01:47:04.434787+0000","last_active":"2024-04-23T01:48:20.861150+0000","last_peered":"2024-04-23T01:48:20.861150+0000","last_clean":"2024-04-23T01:48:20.861150+0000","last_became_active":"2024-04-23T01:47:04.423352+0000","last_became_peered":"2024-04-23T01:47:04.423352+0000","last_unstale":"2024-04-23T01:48:20.861150+0000","last_undegraded":"2024-04-23T01:48:20.861150+0000","last_fullsized":"2024-04-23T01:48:20.861150+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6356992,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":776,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.602643+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:47:05.602643+0000","last_peered":"2024-04-23T01:47:05.602643+0000","last_clean":"2024-04-23T01:47:05.602643+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:47:05.602643+0000","last_undegraded":"2024-04-23T01:47:05.602643+0000","last_fullsized":"2024-04-23T01:47:05.602643+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"29'614","reported_seq":627,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:21.566361+0000","last_change":"2024-04-23T01:47:04.425726+0000","last_active":"2024-04-23T01:48:21.566361+0000","last_peered":"2024-04-23T01:48:21.566361+0000","last_clean":"2024-04-23T01:48:21.566361+0000","last_became_active":"2024-04-23T01:47:04.416905+0000","last_became_peered":"2024-04-23T01:47:04.416905+0000","last_unstale":"2024-04-23T01:48:21.566361+0000","last_undegraded":"2024-04-23T01:48:21.566361+0000","last_fullsized":"2024-04-23T01:48:21.566361+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5038080,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":615,"num_write_kb":4920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.400601+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:47:05.400601+0000","last_peered":"2024-04-23T01:47:05.400601+0000","last_clean":"2024-04-23T01:47:05.400601+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:47:05.400601+0000","last_undegraded":"2024-04-23T01:47:05.400601+0000","last_fullsized":"2024-04-23T01:47:05.400601+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":92553216,"num_objects":1413,"num_object_clones":0,"num_object_copies":4239,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11298,"num_write_kb":90384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":138792960,"data_stored":277585920,"data_compressed":14330022,"data_compressed_allocated":138792960,"data_compressed_original":277585920,"omap_allocated":0,"internal_metadata":0},"log_size":11282,"ondisk_log_size":11282,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705858,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":85104,"kb_used_data":36068,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94286736,"statfs":{"total":96636764160,"available":96549617664,"internally_reserved":0,"allocated":36933632,"data_stored":73924408,"data_compressed":3811557,"data_compressed_allocated":36667392,"data_compressed_original":73826304,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705858,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77468,"kb_used_data":28436,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94294372,"statfs":{"total":96636764160,"available":96557436928,"internally_reserved":0,"allocated":29118464,"data_stored":58294072,"data_compressed":3004606,"data_compressed_allocated":28852224,"data_compressed_original":58195968,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705858,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":81764,"kb_used_data":36892,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94290076,"statfs":{"total":96636764160,"available":96553037824,"internally_reserved":0,"allocated":37777408,"data_stored":75144472,"data_compressed":3875054,"data_compressed_allocated":37531648,"data_compressed_original":75063296,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705858,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":87036,"kb_used_data":33844,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94284804,"statfs":{"total":96636764160,"available":96547639296,"internally_reserved":0,"allocated":34656256,"data_stored":69369656,"data_compressed":3576384,"data_compressed_allocated":34390016,"data_compressed_original":69271552,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":34717696,"data_stored":69435392,"data_compressed":3584501,"data_compressed_allocated":34717696,"data_compressed_original":69435392,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":37982208,"data_stored":75964416,"data_compressed":3921584,"data_compressed_allocated":37982208,"data_compressed_original":75964416,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":29065216,"data_stored":58130432,"data_compressed":3000879,"data_compressed_allocated":29065216,"data_compressed_original":58130432,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":37027840,"data_stored":74055680,"data_compressed":3823058,"data_compressed_allocated":37027840,"data_compressed_original":74055680,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:48:24.387 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 79 16 11673 11657 1.15269 0.984375 0.26377 0.108318 2024-04-23T01:48:24.393 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:48:24.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:24.528+0000 7efdcd66d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:24.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:24.536+0000 7efdcd66d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:25.387 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:48:25.389025+0000 min lat: 0.0142109 max lat: 0.398981 avg lat: 0.108566 2024-04-23T01:48:25.388 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:48:25.388 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 80 16 11794 11778 1.1501 0.945312 0.277815 0.108566 2024-04-23T01:48:26.387 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 81 16 11927 11911 1.14872 1.03906 0.0908553 0.108723 2024-04-23T01:48:26.796 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:48:26.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:26.924+0000 7f9d8d42d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:26.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:26.931+0000 7f9d8d42d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:27.387 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 82 15 12061 12046 1.14758 1.05469 0.250225 0.108806 2024-04-23T01:48:28.388 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 83 15 12189 12174 1.1458 1 0.115065 0.108969 2024-04-23T01:48:29.108 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:48:29.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:29.226+0000 7f5bcb6ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:29.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:29.235+0000 7f5bcb6ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:29.332 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:48:29.332 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:48:29.332 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:48:29.388 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 84 16 12323 12307 1.14453 1.03906 0.290511 0.109108 2024-04-23T01:48:29.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:29.468+0000 7fd80ba6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:29.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:29.478+0000 7fd80ba6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:29.764 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:48:29.764 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:48:29.784 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":124,"stamp":"2024-04-23T01:48:29.587317+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":98411040,"num_objects":1496,"num_object_clones":0,"num_object_copies":4488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":12078,"num_write_kb":97422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12022,"ondisk_log_size":12022,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":347608,"kb_used_data":143160,"kb_used_omap":0,"kb_used_meta":204416,"kb_avail":377139752,"statfs":{"total":386547056640,"available":386191106048,"internally_reserved":0,"allocated":146595840,"data_stored":292952768,"data_compressed":15105141,"data_compressed_allocated":145551360,"data_compressed_original":292577280,"omap_allocated":0,"internal_metadata":209321984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11026432,"num_objects":169,"num_object_clones":0,"num_object_copies":507,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1346,"num_write_kb":10768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002319"},"pg_stats":[{"pgid":"4.f","version":"29'727","reported_seq":740,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:25.591694+0000","last_change":"2024-04-23T01:47:04.491534+0000","last_active":"2024-04-23T01:48:25.591694+0000","last_peered":"2024-04-23T01:48:25.591694+0000","last_clean":"2024-04-23T01:48:25.591694+0000","last_became_active":"2024-04-23T01:47:04.486910+0000","last_became_peered":"2024-04-23T01:47:04.486910+0000","last_unstale":"2024-04-23T01:48:25.591694+0000","last_undegraded":"2024-04-23T01:48:25.591694+0000","last_fullsized":"2024-04-23T01:48:25.591694+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":728,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"29'751","reported_seq":764,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:26.439823+0000","last_change":"2024-04-23T01:47:04.452063+0000","last_active":"2024-04-23T01:48:26.439823+0000","last_peered":"2024-04-23T01:48:26.439823+0000","last_clean":"2024-04-23T01:48:26.439823+0000","last_became_active":"2024-04-23T01:47:04.443666+0000","last_became_peered":"2024-04-23T01:47:04.443666+0000","last_unstale":"2024-04-23T01:48:26.439823+0000","last_undegraded":"2024-04-23T01:48:26.439823+0000","last_fullsized":"2024-04-23T01:48:26.439823+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"29'747","reported_seq":760,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:26.575543+0000","last_change":"2024-04-23T01:47:04.485140+0000","last_active":"2024-04-23T01:48:26.575543+0000","last_peered":"2024-04-23T01:48:26.575543+0000","last_clean":"2024-04-23T01:48:26.575543+0000","last_became_active":"2024-04-23T01:47:04.480664+0000","last_became_peered":"2024-04-23T01:47:04.480664+0000","last_unstale":"2024-04-23T01:48:26.575543+0000","last_undegraded":"2024-04-23T01:48:26.575543+0000","last_fullsized":"2024-04-23T01:48:26.575543+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":747,"log_dups_size":0,"ondisk_log_size":747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6127616,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":748,"num_write_kb":5984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"29'767","reported_seq":780,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:25.985372+0000","last_change":"2024-04-23T01:47:04.442089+0000","last_active":"2024-04-23T01:48:25.985372+0000","last_peered":"2024-04-23T01:48:25.985372+0000","last_clean":"2024-04-23T01:48:25.985372+0000","last_became_active":"2024-04-23T01:47:04.431317+0000","last_became_peered":"2024-04-23T01:47:04.431317+0000","last_unstale":"2024-04-23T01:48:25.985372+0000","last_undegraded":"2024-04-23T01:48:25.985372+0000","last_fullsized":"2024-04-23T01:48:25.985372+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":768,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"29'775","reported_seq":788,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:25.609580+0000","last_change":"2024-04-23T01:47:04.503686+0000","last_active":"2024-04-23T01:48:25.609580+0000","last_peered":"2024-04-23T01:48:25.609580+0000","last_clean":"2024-04-23T01:48:25.609580+0000","last_became_active":"2024-04-23T01:47:04.499155+0000","last_became_peered":"2024-04-23T01:47:04.499155+0000","last_unstale":"2024-04-23T01:48:25.609580+0000","last_undegraded":"2024-04-23T01:48:25.609580+0000","last_fullsized":"2024-04-23T01:48:25.609580+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6356992,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":776,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"29'903","reported_seq":916,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:26.233551+0000","last_change":"2024-04-23T01:47:04.516672+0000","last_active":"2024-04-23T01:48:26.233551+0000","last_peered":"2024-04-23T01:48:26.233551+0000","last_clean":"2024-04-23T01:48:26.233551+0000","last_became_active":"2024-04-23T01:47:04.504711+0000","last_became_peered":"2024-04-23T01:47:04.504711+0000","last_unstale":"2024-04-23T01:48:26.233551+0000","last_undegraded":"2024-04-23T01:48:26.233551+0000","last_fullsized":"2024-04-23T01:48:26.233551+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":903,"log_dups_size":0,"ondisk_log_size":903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7405568,"num_objects":113,"num_object_clones":0,"num_object_copies":339,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":904,"num_write_kb":7232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"29'647","reported_seq":660,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:26.192490+0000","last_change":"2024-04-23T01:47:04.449451+0000","last_active":"2024-04-23T01:48:26.192490+0000","last_peered":"2024-04-23T01:48:26.192490+0000","last_clean":"2024-04-23T01:48:26.192490+0000","last_became_active":"2024-04-23T01:47:04.442623+0000","last_became_peered":"2024-04-23T01:47:04.442623+0000","last_unstale":"2024-04-23T01:48:26.192490+0000","last_undegraded":"2024-04-23T01:48:26.192490+0000","last_fullsized":"2024-04-23T01:48:26.192490+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308416,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":648,"num_write_kb":5184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"29'735","reported_seq":748,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:26.115093+0000","last_change":"2024-04-23T01:47:04.520179+0000","last_active":"2024-04-23T01:48:26.115093+0000","last_peered":"2024-04-23T01:48:26.115093+0000","last_clean":"2024-04-23T01:48:26.115093+0000","last_became_active":"2024-04-23T01:47:04.511239+0000","last_became_peered":"2024-04-23T01:47:04.511239+0000","last_unstale":"2024-04-23T01:48:26.115093+0000","last_undegraded":"2024-04-23T01:48:26.115093+0000","last_fullsized":"2024-04-23T01:48:26.115093+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6029312,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":736,"num_write_kb":5888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"29'767","reported_seq":780,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:26.565382+0000","last_change":"2024-04-23T01:47:04.496716+0000","last_active":"2024-04-23T01:48:26.565382+0000","last_peered":"2024-04-23T01:48:26.565382+0000","last_clean":"2024-04-23T01:48:26.565382+0000","last_became_active":"2024-04-23T01:47:04.486607+0000","last_became_peered":"2024-04-23T01:47:04.486607+0000","last_unstale":"2024-04-23T01:48:26.565382+0000","last_undegraded":"2024-04-23T01:48:26.565382+0000","last_fullsized":"2024-04-23T01:48:26.565382+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":768,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":186,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.325282+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:47:05.325282+0000","last_peered":"2024-04-23T01:47:05.325282+0000","last_clean":"2024-04-23T01:47:05.325282+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:47:05.325282+0000","last_undegraded":"2024-04-23T01:47:05.325282+0000","last_fullsized":"2024-04-23T01:47:05.325282+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.410555+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:47:05.410555+0000","last_peered":"2024-04-23T01:47:05.410555+0000","last_clean":"2024-04-23T01:47:05.410555+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:47:05.410555+0000","last_undegraded":"2024-04-23T01:47:05.410555+0000","last_fullsized":"2024-04-23T01:47:05.410555+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"29'727","reported_seq":740,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:25.724666+0000","last_change":"2024-04-23T01:47:04.467386+0000","last_active":"2024-04-23T01:48:25.724666+0000","last_peered":"2024-04-23T01:48:25.724666+0000","last_clean":"2024-04-23T01:48:25.724666+0000","last_became_active":"2024-04-23T01:47:04.459042+0000","last_became_peered":"2024-04-23T01:47:04.459042+0000","last_unstale":"2024-04-23T01:48:25.724666+0000","last_undegraded":"2024-04-23T01:48:25.724666+0000","last_fullsized":"2024-04-23T01:48:25.724666+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":728,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.570797+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:47:05.570797+0000","last_peered":"2024-04-23T01:47:05.570797+0000","last_clean":"2024-04-23T01:47:05.570797+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:47:05.570797+0000","last_undegraded":"2024-04-23T01:47:05.570797+0000","last_fullsized":"2024-04-23T01:47:05.570797+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"29'671","reported_seq":684,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:25.522622+0000","last_change":"2024-04-23T01:47:04.466464+0000","last_active":"2024-04-23T01:48:25.522622+0000","last_peered":"2024-04-23T01:48:25.522622+0000","last_clean":"2024-04-23T01:48:25.522622+0000","last_became_active":"2024-04-23T01:47:04.460338+0000","last_became_peered":"2024-04-23T01:47:04.460338+0000","last_unstale":"2024-04-23T01:48:25.522622+0000","last_undegraded":"2024-04-23T01:48:25.522622+0000","last_fullsized":"2024-04-23T01:48:25.522622+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505024,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":672,"num_write_kb":5376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.397538+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:47:05.397538+0000","last_peered":"2024-04-23T01:47:05.397538+0000","last_clean":"2024-04-23T01:47:05.397538+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:47:05.397538+0000","last_undegraded":"2024-04-23T01:47:05.397538+0000","last_fullsized":"2024-04-23T01:47:05.397538+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"29'719","reported_seq":732,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:26.325098+0000","last_change":"2024-04-23T01:47:04.435560+0000","last_active":"2024-04-23T01:48:26.325098+0000","last_peered":"2024-04-23T01:48:26.325098+0000","last_clean":"2024-04-23T01:48:26.325098+0000","last_became_active":"2024-04-23T01:47:04.424493+0000","last_became_peered":"2024-04-23T01:47:04.424493+0000","last_unstale":"2024-04-23T01:48:26.325098+0000","last_undegraded":"2024-04-23T01:48:26.325098+0000","last_fullsized":"2024-04-23T01:48:26.325098+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":720,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.576933+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:47:05.576933+0000","last_peered":"2024-04-23T01:47:05.576933+0000","last_clean":"2024-04-23T01:47:05.576933+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:47:05.576933+0000","last_undegraded":"2024-04-23T01:47:05.576933+0000","last_fullsized":"2024-04-23T01:47:05.576933+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"29'679","reported_seq":692,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:26.474789+0000","last_change":"2024-04-23T01:47:04.530152+0000","last_active":"2024-04-23T01:48:26.474789+0000","last_peered":"2024-04-23T01:48:26.474789+0000","last_clean":"2024-04-23T01:48:26.474789+0000","last_became_active":"2024-04-23T01:47:04.519844+0000","last_became_peered":"2024-04-23T01:47:04.519844+0000","last_unstale":"2024-04-23T01:48:26.474789+0000","last_undegraded":"2024-04-23T01:48:26.474789+0000","last_fullsized":"2024-04-23T01:48:26.474789+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5570560,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":680,"num_write_kb":5440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.590971+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:47:05.590971+0000","last_peered":"2024-04-23T01:47:05.590971+0000","last_clean":"2024-04-23T01:47:05.590971+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:47:05.590971+0000","last_undegraded":"2024-04-23T01:47:05.590971+0000","last_fullsized":"2024-04-23T01:47:05.590971+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"29'861","reported_seq":874,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:26.563554+0000","last_change":"2024-04-23T01:47:04.465832+0000","last_active":"2024-04-23T01:48:26.563554+0000","last_peered":"2024-04-23T01:48:26.563554+0000","last_clean":"2024-04-23T01:48:26.563554+0000","last_became_active":"2024-04-23T01:47:04.460695+0000","last_became_peered":"2024-04-23T01:47:04.460695+0000","last_unstale":"2024-04-23T01:48:26.563554+0000","last_undegraded":"2024-04-23T01:48:26.563554+0000","last_fullsized":"2024-04-23T01:48:26.563554+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":861,"log_dups_size":0,"ondisk_log_size":861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7061504,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":862,"num_write_kb":6896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.331332+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:47:05.331332+0000","last_peered":"2024-04-23T01:47:05.331332+0000","last_clean":"2024-04-23T01:47:05.331332+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:47:05.331332+0000","last_undegraded":"2024-04-23T01:47:05.331332+0000","last_fullsized":"2024-04-23T01:47:05.331332+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"29'794","reported_seq":807,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:26.568468+0000","last_change":"2024-04-23T01:47:04.434787+0000","last_active":"2024-04-23T01:48:26.568468+0000","last_peered":"2024-04-23T01:48:26.568468+0000","last_clean":"2024-04-23T01:48:26.568468+0000","last_became_active":"2024-04-23T01:47:04.423352+0000","last_became_peered":"2024-04-23T01:47:04.423352+0000","last_unstale":"2024-04-23T01:48:26.568468+0000","last_undegraded":"2024-04-23T01:48:26.568468+0000","last_fullsized":"2024-04-23T01:48:26.568468+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":794,"log_dups_size":0,"ondisk_log_size":794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6512640,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":795,"num_write_kb":6360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.602643+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:47:05.602643+0000","last_peered":"2024-04-23T01:47:05.602643+0000","last_clean":"2024-04-23T01:47:05.602643+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:47:05.602643+0000","last_undegraded":"2024-04-23T01:47:05.602643+0000","last_fullsized":"2024-04-23T01:47:05.602643+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"29'655","reported_seq":668,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:24.320549+0000","last_change":"2024-04-23T01:47:04.425726+0000","last_active":"2024-04-23T01:48:24.320549+0000","last_peered":"2024-04-23T01:48:24.320549+0000","last_clean":"2024-04-23T01:48:24.320549+0000","last_became_active":"2024-04-23T01:47:04.416905+0000","last_became_peered":"2024-04-23T01:47:04.416905+0000","last_unstale":"2024-04-23T01:48:24.320549+0000","last_undegraded":"2024-04-23T01:48:24.320549+0000","last_fullsized":"2024-04-23T01:48:24.320549+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.400601+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:47:05.400601+0000","last_peered":"2024-04-23T01:47:05.400601+0000","last_clean":"2024-04-23T01:47:05.400601+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:47:05.400601+0000","last_undegraded":"2024-04-23T01:47:05.400601+0000","last_fullsized":"2024-04-23T01:47:05.400601+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":97820672,"num_objects":1494,"num_object_clones":0,"num_object_copies":4482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11941,"num_write_kb":95528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":146710528,"data_stored":293421056,"data_compressed":15147681,"data_compressed_allocated":146710528,"data_compressed_original":293421056,"omap_allocated":0,"internal_metadata":0},"log_size":11925,"ondisk_log_size":11925,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":91408,"kb_used_data":38216,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94280432,"statfs":{"total":96636764160,"available":96543162368,"internally_reserved":0,"allocated":39133184,"data_stored":78323512,"data_compressed":4038708,"data_compressed_allocated":38866944,"data_compressed_original":78225408,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":79316,"kb_used_data":30284,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94292524,"statfs":{"total":96636764160,"available":96555544576,"internally_reserved":0,"allocated":31010816,"data_stored":62078776,"data_compressed":3200032,"data_compressed_allocated":30744576,"data_compressed_original":61980672,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":83932,"kb_used_data":39060,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94287908,"statfs":{"total":96636764160,"available":96550817792,"internally_reserved":0,"allocated":39997440,"data_stored":79584536,"data_compressed":4104320,"data_compressed_allocated":39751680,"data_compressed_original":79503360,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":92952,"kb_used_data":35600,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94278888,"statfs":{"total":96636764160,"available":96541581312,"internally_reserved":0,"allocated":36454400,"data_stored":72965944,"data_compressed":3762081,"data_compressed_allocated":36188160,"data_compressed_original":72867840,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36388864,"data_stored":72777728,"data_compressed":3757085,"data_compressed_allocated":36388864,"data_compressed_original":72777728,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40194048,"data_stored":80388096,"data_compressed":4150004,"data_compressed_allocated":40194048,"data_compressed_original":80388096,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":30998528,"data_stored":61997056,"data_compressed":3200535,"data_compressed_allocated":30998528,"data_compressed_original":61997056,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":39129088,"data_stored":78258176,"data_compressed":4040057,"data_compressed_allocated":39129088,"data_compressed_original":78258176,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:48:30.388 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 85 16 12467 12451 1.1443 1.125 0.0846027 0.109141 2024-04-23T01:48:31.388 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 86 16 12604 12588 1.14343 1.07031 0.101474 0.109215 2024-04-23T01:48:31.423 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:48:31.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:31.567+0000 7f5e532ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:31.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:31.574+0000 7f5e532ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:32.388 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 87 16 12745 12729 1.14295 1.10156 0.112849 0.109288 2024-04-23T01:48:33.390 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 88 16 12884 12868 1.1423 1.08594 0.0619062 0.109337 2024-04-23T01:48:33.767 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:48:33.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:33.921+0000 7f8fa651d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:33.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:33.929+0000 7f8fa651d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:34.388 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 89 16 13009 12993 1.14044 0.976562 0.129155 0.109524 2024-04-23T01:48:34.787 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:48:34.787 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:48:34.788 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:48:34.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:34.912+0000 7f5e28c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:34.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:34.923+0000 7f5e28c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:35.150 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:48:35.151 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:48:35.169 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2024-04-23T01:48:33.588050+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":103973408,"num_objects":1581,"num_object_clones":0,"num_object_copies":4743,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":12757,"num_write_kb":102854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12701,"ondisk_log_size":12701,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":368284,"kb_used_data":151356,"kb_used_omap":0,"kb_used_meta":216896,"kb_avail":377119076,"statfs":{"total":386547056640,"available":386169933824,"internally_reserved":0,"allocated":154988544,"data_stored":309738176,"data_compressed":15971868,"data_compressed_allocated":153944064,"data_compressed_original":309362688,"omap_allocated":0,"internal_metadata":222101504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16588800,"num_objects":254,"num_object_clones":0,"num_object_copies":762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2025,"num_write_kb":16200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002294"},"pg_stats":[{"pgid":"4.f","version":"29'779","reported_seq":792,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:31.575157+0000","last_change":"2024-04-23T01:47:04.491534+0000","last_active":"2024-04-23T01:48:31.575157+0000","last_peered":"2024-04-23T01:48:31.575157+0000","last_clean":"2024-04-23T01:48:31.575157+0000","last_became_active":"2024-04-23T01:47:04.486910+0000","last_became_peered":"2024-04-23T01:47:04.486910+0000","last_unstale":"2024-04-23T01:48:31.575157+0000","last_undegraded":"2024-04-23T01:48:31.575157+0000","last_fullsized":"2024-04-23T01:48:31.575157+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":779,"log_dups_size":0,"ondisk_log_size":779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6389760,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":780,"num_write_kb":6240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"29'799","reported_seq":812,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:30.698997+0000","last_change":"2024-04-23T01:47:04.452063+0000","last_active":"2024-04-23T01:48:30.698997+0000","last_peered":"2024-04-23T01:48:30.698997+0000","last_clean":"2024-04-23T01:48:30.698997+0000","last_became_active":"2024-04-23T01:47:04.443666+0000","last_became_peered":"2024-04-23T01:47:04.443666+0000","last_unstale":"2024-04-23T01:48:30.698997+0000","last_undegraded":"2024-04-23T01:48:30.698997+0000","last_fullsized":"2024-04-23T01:48:30.698997+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":799,"log_dups_size":0,"ondisk_log_size":799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6553600,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":800,"num_write_kb":6400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"29'791","reported_seq":804,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:29.891726+0000","last_change":"2024-04-23T01:47:04.485140+0000","last_active":"2024-04-23T01:48:29.891726+0000","last_peered":"2024-04-23T01:48:29.891726+0000","last_clean":"2024-04-23T01:48:29.891726+0000","last_became_active":"2024-04-23T01:47:04.480664+0000","last_became_peered":"2024-04-23T01:47:04.480664+0000","last_unstale":"2024-04-23T01:48:29.891726+0000","last_undegraded":"2024-04-23T01:48:29.891726+0000","last_fullsized":"2024-04-23T01:48:29.891726+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6488064,"num_objects":99,"num_object_clones":0,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":792,"num_write_kb":6336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"29'815","reported_seq":828,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:31.509787+0000","last_change":"2024-04-23T01:47:04.442089+0000","last_active":"2024-04-23T01:48:31.509787+0000","last_peered":"2024-04-23T01:48:31.509787+0000","last_clean":"2024-04-23T01:48:31.509787+0000","last_became_active":"2024-04-23T01:47:04.431317+0000","last_became_peered":"2024-04-23T01:47:04.431317+0000","last_unstale":"2024-04-23T01:48:31.509787+0000","last_undegraded":"2024-04-23T01:48:31.509787+0000","last_fullsized":"2024-04-23T01:48:31.509787+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6684672,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":816,"num_write_kb":6528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"29'808","reported_seq":821,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:29.995876+0000","last_change":"2024-04-23T01:47:04.503686+0000","last_active":"2024-04-23T01:48:29.995876+0000","last_peered":"2024-04-23T01:48:29.995876+0000","last_clean":"2024-04-23T01:48:29.995876+0000","last_became_active":"2024-04-23T01:47:04.499155+0000","last_became_peered":"2024-04-23T01:47:04.499155+0000","last_unstale":"2024-04-23T01:48:29.995876+0000","last_undegraded":"2024-04-23T01:48:29.995876+0000","last_fullsized":"2024-04-23T01:48:29.995876+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":808,"log_dups_size":0,"ondisk_log_size":808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6627328,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":809,"num_write_kb":6472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"29'943","reported_seq":956,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:30.304659+0000","last_change":"2024-04-23T01:47:04.516672+0000","last_active":"2024-04-23T01:48:30.304659+0000","last_peered":"2024-04-23T01:48:30.304659+0000","last_clean":"2024-04-23T01:48:30.304659+0000","last_became_active":"2024-04-23T01:47:04.504711+0000","last_became_peered":"2024-04-23T01:47:04.504711+0000","last_unstale":"2024-04-23T01:48:30.304659+0000","last_undegraded":"2024-04-23T01:48:30.304659+0000","last_fullsized":"2024-04-23T01:48:30.304659+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":943,"log_dups_size":0,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7733248,"num_objects":118,"num_object_clones":0,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":944,"num_write_kb":7552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"29'695","reported_seq":708,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:31.280800+0000","last_change":"2024-04-23T01:47:04.449451+0000","last_active":"2024-04-23T01:48:31.280800+0000","last_peered":"2024-04-23T01:48:31.280800+0000","last_clean":"2024-04-23T01:48:31.280800+0000","last_became_active":"2024-04-23T01:47:04.442623+0000","last_became_peered":"2024-04-23T01:47:04.442623+0000","last_unstale":"2024-04-23T01:48:31.280800+0000","last_undegraded":"2024-04-23T01:48:31.280800+0000","last_fullsized":"2024-04-23T01:48:31.280800+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":696,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"29'791","reported_seq":804,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:31.402344+0000","last_change":"2024-04-23T01:47:04.520179+0000","last_active":"2024-04-23T01:48:31.402344+0000","last_peered":"2024-04-23T01:48:31.402344+0000","last_clean":"2024-04-23T01:48:31.402344+0000","last_became_active":"2024-04-23T01:47:04.511239+0000","last_became_peered":"2024-04-23T01:47:04.511239+0000","last_unstale":"2024-04-23T01:48:31.402344+0000","last_undegraded":"2024-04-23T01:48:31.402344+0000","last_fullsized":"2024-04-23T01:48:31.402344+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6488064,"num_objects":99,"num_object_clones":0,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":792,"num_write_kb":6336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"29'822","reported_seq":835,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:31.573584+0000","last_change":"2024-04-23T01:47:04.496716+0000","last_active":"2024-04-23T01:48:31.573584+0000","last_peered":"2024-04-23T01:48:31.573584+0000","last_clean":"2024-04-23T01:48:31.573584+0000","last_became_active":"2024-04-23T01:47:04.486607+0000","last_became_peered":"2024-04-23T01:47:04.486607+0000","last_unstale":"2024-04-23T01:48:31.573584+0000","last_undegraded":"2024-04-23T01:48:31.573584+0000","last_fullsized":"2024-04-23T01:48:31.573584+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":822,"log_dups_size":0,"ondisk_log_size":822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6742016,"num_objects":103,"num_object_clones":0,"num_object_copies":309,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":823,"num_write_kb":6584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":186,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.325282+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:47:05.325282+0000","last_peered":"2024-04-23T01:47:05.325282+0000","last_clean":"2024-04-23T01:47:05.325282+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:47:05.325282+0000","last_undegraded":"2024-04-23T01:47:05.325282+0000","last_fullsized":"2024-04-23T01:47:05.325282+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.410555+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:47:05.410555+0000","last_peered":"2024-04-23T01:47:05.410555+0000","last_clean":"2024-04-23T01:47:05.410555+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:47:05.410555+0000","last_undegraded":"2024-04-23T01:47:05.410555+0000","last_fullsized":"2024-04-23T01:47:05.410555+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"29'727","reported_seq":740,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:25.724666+0000","last_change":"2024-04-23T01:47:04.467386+0000","last_active":"2024-04-23T01:48:25.724666+0000","last_peered":"2024-04-23T01:48:25.724666+0000","last_clean":"2024-04-23T01:48:25.724666+0000","last_became_active":"2024-04-23T01:47:04.459042+0000","last_became_peered":"2024-04-23T01:47:04.459042+0000","last_unstale":"2024-04-23T01:48:25.724666+0000","last_undegraded":"2024-04-23T01:48:25.724666+0000","last_fullsized":"2024-04-23T01:48:25.724666+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":728,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.570797+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:47:05.570797+0000","last_peered":"2024-04-23T01:47:05.570797+0000","last_clean":"2024-04-23T01:47:05.570797+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:47:05.570797+0000","last_undegraded":"2024-04-23T01:47:05.570797+0000","last_fullsized":"2024-04-23T01:47:05.570797+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"29'735","reported_seq":748,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:30.857389+0000","last_change":"2024-04-23T01:47:04.466464+0000","last_active":"2024-04-23T01:48:30.857389+0000","last_peered":"2024-04-23T01:48:30.857389+0000","last_clean":"2024-04-23T01:48:30.857389+0000","last_became_active":"2024-04-23T01:47:04.460338+0000","last_became_peered":"2024-04-23T01:47:04.460338+0000","last_unstale":"2024-04-23T01:48:30.857389+0000","last_undegraded":"2024-04-23T01:48:30.857389+0000","last_fullsized":"2024-04-23T01:48:30.857389+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6029312,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":736,"num_write_kb":5888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.397538+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:47:05.397538+0000","last_peered":"2024-04-23T01:47:05.397538+0000","last_clean":"2024-04-23T01:47:05.397538+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:47:05.397538+0000","last_undegraded":"2024-04-23T01:47:05.397538+0000","last_fullsized":"2024-04-23T01:47:05.397538+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"29'743","reported_seq":756,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:31.095909+0000","last_change":"2024-04-23T01:47:04.435560+0000","last_active":"2024-04-23T01:48:31.095909+0000","last_peered":"2024-04-23T01:48:31.095909+0000","last_clean":"2024-04-23T01:48:31.095909+0000","last_became_active":"2024-04-23T01:47:04.424493+0000","last_became_peered":"2024-04-23T01:47:04.424493+0000","last_unstale":"2024-04-23T01:48:31.095909+0000","last_undegraded":"2024-04-23T01:48:31.095909+0000","last_fullsized":"2024-04-23T01:48:31.095909+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6094848,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":744,"num_write_kb":5952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.576933+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:47:05.576933+0000","last_peered":"2024-04-23T01:47:05.576933+0000","last_clean":"2024-04-23T01:47:05.576933+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:47:05.576933+0000","last_undegraded":"2024-04-23T01:47:05.576933+0000","last_fullsized":"2024-04-23T01:47:05.576933+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"29'727","reported_seq":740,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:29.738839+0000","last_change":"2024-04-23T01:47:04.530152+0000","last_active":"2024-04-23T01:48:29.738839+0000","last_peered":"2024-04-23T01:48:29.738839+0000","last_clean":"2024-04-23T01:48:29.738839+0000","last_became_active":"2024-04-23T01:47:04.519844+0000","last_became_peered":"2024-04-23T01:47:04.519844+0000","last_unstale":"2024-04-23T01:48:29.738839+0000","last_undegraded":"2024-04-23T01:48:29.738839+0000","last_fullsized":"2024-04-23T01:48:29.738839+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":728,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.590971+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:47:05.590971+0000","last_peered":"2024-04-23T01:47:05.590971+0000","last_clean":"2024-04-23T01:47:05.590971+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:47:05.590971+0000","last_undegraded":"2024-04-23T01:47:05.590971+0000","last_fullsized":"2024-04-23T01:47:05.590971+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"29'895","reported_seq":908,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:30.706125+0000","last_change":"2024-04-23T01:47:04.465832+0000","last_active":"2024-04-23T01:48:30.706125+0000","last_peered":"2024-04-23T01:48:30.706125+0000","last_clean":"2024-04-23T01:48:30.706125+0000","last_became_active":"2024-04-23T01:47:04.460695+0000","last_became_peered":"2024-04-23T01:47:04.460695+0000","last_unstale":"2024-04-23T01:48:30.706125+0000","last_undegraded":"2024-04-23T01:48:30.706125+0000","last_fullsized":"2024-04-23T01:48:30.706125+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340032,"num_objects":112,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":896,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.331332+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:47:05.331332+0000","last_peered":"2024-04-23T01:47:05.331332+0000","last_clean":"2024-04-23T01:47:05.331332+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:47:05.331332+0000","last_undegraded":"2024-04-23T01:47:05.331332+0000","last_fullsized":"2024-04-23T01:47:05.331332+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"29'863","reported_seq":876,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:31.280063+0000","last_change":"2024-04-23T01:47:04.434787+0000","last_active":"2024-04-23T01:48:31.280063+0000","last_peered":"2024-04-23T01:48:31.280063+0000","last_clean":"2024-04-23T01:48:31.280063+0000","last_became_active":"2024-04-23T01:47:04.423352+0000","last_became_peered":"2024-04-23T01:47:04.423352+0000","last_unstale":"2024-04-23T01:48:31.280063+0000","last_undegraded":"2024-04-23T01:48:31.280063+0000","last_fullsized":"2024-04-23T01:48:31.280063+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":863,"log_dups_size":0,"ondisk_log_size":863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":864,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.602643+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:47:05.602643+0000","last_peered":"2024-04-23T01:47:05.602643+0000","last_clean":"2024-04-23T01:47:05.602643+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:47:05.602643+0000","last_undegraded":"2024-04-23T01:47:05.602643+0000","last_fullsized":"2024-04-23T01:47:05.602643+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"29'671","reported_seq":684,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:48:28.927789+0000","last_change":"2024-04-23T01:47:04.425726+0000","last_active":"2024-04-23T01:48:28.927789+0000","last_peered":"2024-04-23T01:48:28.927789+0000","last_clean":"2024-04-23T01:48:28.927789+0000","last_became_active":"2024-04-23T01:47:04.416905+0000","last_became_peered":"2024-04-23T01:47:04.416905+0000","last_unstale":"2024-04-23T01:48:28.927789+0000","last_undegraded":"2024-04-23T01:48:28.927789+0000","last_fullsized":"2024-04-23T01:48:28.927789+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:47:03.285808+0000","last_clean_scrub_stamp":"2024-04-23T01:47:03.285808+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505024,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":672,"num_write_kb":5376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.400601+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:47:05.400601+0000","last_peered":"2024-04-23T01:47:05.400601+0000","last_clean":"2024-04-23T01:47:05.400601+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:47:05.400601+0000","last_undegraded":"2024-04-23T01:47:05.400601+0000","last_fullsized":"2024-04-23T01:47:05.400601+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":103383040,"num_objects":1579,"num_object_clones":0,"num_object_copies":4737,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12620,"num_write_kb":100960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155066368,"data_stored":310132736,"data_compressed":16010601,"data_compressed_allocated":155066368,"data_compressed_original":310132736,"omap_allocated":0,"internal_metadata":0},"log_size":12604,"ondisk_log_size":12604,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":97732,"kb_used_data":40380,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94274108,"statfs":{"total":96636764160,"available":96536686592,"internally_reserved":0,"allocated":41349120,"data_stored":82755384,"data_compressed":4267551,"data_compressed_allocated":41082880,"data_compressed_original":82657280,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":85184,"kb_used_data":31992,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94286656,"statfs":{"total":96636764160,"available":96549535744,"internally_reserved":0,"allocated":32759808,"data_stored":65576760,"data_compressed":3380653,"data_compressed_allocated":32493568,"data_compressed_original":65478656,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":90452,"kb_used_data":41420,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94281388,"statfs":{"total":96636764160,"available":96544141312,"internally_reserved":0,"allocated":42414080,"data_stored":84417816,"data_compressed":4353890,"data_compressed_allocated":42168320,"data_compressed_original":84336640,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":94916,"kb_used_data":37564,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94276924,"statfs":{"total":96636764160,"available":96539570176,"internally_reserved":0,"allocated":38465536,"data_stored":76988216,"data_compressed":3969774,"data_compressed_allocated":38199296,"data_compressed_original":76890112,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":38518784,"data_stored":77037568,"data_compressed":3977045,"data_compressed_allocated":38518784,"data_compressed_original":77037568,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":42504192,"data_stored":85008384,"data_compressed":4388576,"data_compressed_allocated":42504192,"data_compressed_original":85008384,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32587776,"data_stored":65175552,"data_compressed":3364659,"data_compressed_allocated":32587776,"data_compressed_original":65175552,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":41455616,"data_stored":82911232,"data_compressed":4280321,"data_compressed_allocated":41455616,"data_compressed_original":82911232,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:48:35.388 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 90 16 13141 13125 1.13922 1.03125 0.03387 0.10959 2024-04-23T01:48:36.128 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:48:36.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:36.231+0000 7f8310824640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:36.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:36.237+0000 7f8310824640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:36.389 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Total time run: 90.114 2024-04-23T01:48:36.389 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Total writes made: 13141 2024-04-23T01:48:36.389 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Write size: 8192 2024-04-23T01:48:36.389 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Object size: 65536 2024-04-23T01:48:36.389 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Bandwidth (MB/sec): 1.13927 2024-04-23T01:48:36.389 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Stddev Bandwidth: 0.103917 2024-04-23T01:48:36.389 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Max bandwidth (MB/sec): 1.35938 2024-04-23T01:48:36.389 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Min bandwidth (MB/sec): 0.859375 2024-04-23T01:48:36.389 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Average IOPS: 145 2024-04-23T01:48:36.389 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Stddev IOPS: 13.3014 2024-04-23T01:48:36.389 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Max IOPS: 174 2024-04-23T01:48:36.389 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Min IOPS: 110 2024-04-23T01:48:36.390 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Average Latency(s): 0.10965 2024-04-23T01:48:36.390 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Stddev Latency(s): 0.0569026 2024-04-23T01:48:36.390 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Max latency(s): 0.398981 2024-04-23T01:48:36.390 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Min latency(s): 0.0142109 2024-04-23T01:48:36.410 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Cleaning up (deleting benchmark objects) 2024-04-23T01:48:38.332 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:48:38.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:38.460+0000 7f7043c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:38.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:38.468+0000 7f7043c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:40.172 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:48:40.172 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:48:40.173 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:48:40.173 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:48:40.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:40.330+0000 7f1d0bb9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:40.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:40.340+0000 7f1d0bb9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:40.365 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:48:40.365 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:48:40.365 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:48:40.365 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:48:40.365 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:48:40.366 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:48:40.366 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:48:40.366 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:48:40.366 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:48:40.366 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:48:40.366 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:48:40.366 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:48:40.368 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:48:40.368 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:48:40.369 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:48:40.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:40.525+0000 7f6ce3b5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:40.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:40.536+0000 7f6ce3b5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:40.656 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:48:40.656 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:48:40.656 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:48:40.656 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:48:40.656 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:48:40.656 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:48:40.656 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:48:40.656 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:48:40.657 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:48:40.657 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:48:40.657 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:48:40.657 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:48:40.659 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:48:40.659 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:48:40.659 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:48:40.753 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:48:40.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:40.799+0000 7f0882d5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:40.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:40.816+0000 7f0882d5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:40.839 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:48:40.839 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:48:40.839 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:48:40.839 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:48:40.839 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:48:40.839 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:48:40.839 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:48:40.839 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:48:40.839 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:48:40.839 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:48:40.840 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:48:40.840 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:48:40.842 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:48:40.843 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:48:40.843 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:48:40.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:40.913+0000 7f5e42a0f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:40.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:40.923+0000 7f5e42a0f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:41.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:40.999+0000 7fb614eed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:41.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:41.012+0000 7fb614eed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:41.051 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:48:41.052 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:48:41.052 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:48:41.052 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:48:41.052 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:48:41.052 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:48:41.052 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:48:41.052 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:48:41.052 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:48:41.052 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:48:41.052 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:48:41.052 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:48:41.055 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:48:41.055 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:48:43.353 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:48:43.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:43.484+0000 7f72f155d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:43.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:43.491+0000 7f72f155d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:46.004 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:48:46.056 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:48:46.057 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-23T01:48:46.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:46.144+0000 7f0d7959d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:46.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:46.154+0000 7f0d7959d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:46.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:46.189+0000 7fdc69224640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:46.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:46.205+0000 7fdc69224640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:46.212 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Removed 1643 objects 2024-04-23T01:48:46.212 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Clean up completed and total clean up time :9.80196 2024-04-23T01:48:47.275 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.0 to 1 (10000) 2024-04-23T01:48:47.292 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-23T01:48:47.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:47.401+0000 7f828111d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:47.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:47.410+0000 7f828111d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:48.285 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.1 to 1 (10000) 2024-04-23T01:48:48.285 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:48:48.301 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-23T01:48:48.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:48.393+0000 7f4cee0ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:48.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:48.399+0000 7f4cee0ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:48.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:48.462+0000 7fb935954640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:48.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:48.475+0000 7fb935954640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:49.044 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_1 2024-04-23T01:48:49.045 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_1 unique_pool_1 --yes-i-really-really-mean-it 2024-04-23T01:48:49.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:49.149+0000 7fb1822ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:49.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:49.161+0000 7fb1822ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:49.296 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.2 to 1 (10000) 2024-04-23T01:48:49.313 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-23T01:48:49.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:49.418+0000 7f3172f1d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:49.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:49.428+0000 7f3172f1d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:50.309 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'unique_pool_1' removed 2024-04-23T01:48:50.309 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.3 to 1 (10000) 2024-04-23T01:48:50.324 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-23T01:48:50.325 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:48:50.326 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-04-23T01:48:50.327 INFO:tasks.radosbench:Beginning radosbench... 2024-04-23T01:48:50.327 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_2 2024-04-23T01:48:50.327 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_2 16 2024-04-23T01:48:50.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:50.474+0000 7fc5a506d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:50.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:50.474+0000 7f4443c2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:50.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:50.481+0000 7fc5a506d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:50.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:50.487+0000 7f4443c2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:50.556 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:48:50.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:50.661+0000 7f646c2ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:50.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:50.667+0000 7f646c2ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:50.682 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:48:50.682 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:48:50.695 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2024-04-23T01:48:50.311239+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":214396,"kb_used_data":5756,"kb_used_omap":0,"kb_used_meta":208576,"kb_avail":377272964,"statfs":{"total":386547056640,"available":386327515136,"internally_reserved":0,"allocated":5894144,"data_stored":11524872,"data_compressed":576732,"data_compressed_allocated":4837376,"data_compressed_original":11149312,"omap_allocated":0,"internal_metadata":213581824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-51552280,"num_objects":-788,"num_object_clones":0,"num_object_copies":-2364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-13998,"num_write_kb":-105129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.720569"},"pg_stats":[{"pgid":"1.0","version":"12'97","reported_seq":186,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.325282+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:47:05.325282+0000","last_peered":"2024-04-23T01:47:05.325282+0000","last_clean":"2024-04-23T01:47:05.325282+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:47:05.325282+0000","last_undegraded":"2024-04-23T01:47:05.325282+0000","last_fullsized":"2024-04-23T01:47:05.325282+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.410555+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:47:05.410555+0000","last_peered":"2024-04-23T01:47:05.410555+0000","last_clean":"2024-04-23T01:47:05.410555+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:47:05.410555+0000","last_undegraded":"2024-04-23T01:47:05.410555+0000","last_fullsized":"2024-04-23T01:47:05.410555+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.570797+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:47:05.570797+0000","last_peered":"2024-04-23T01:47:05.570797+0000","last_clean":"2024-04-23T01:47:05.570797+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:47:05.570797+0000","last_undegraded":"2024-04-23T01:47:05.570797+0000","last_fullsized":"2024-04-23T01:47:05.570797+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.397538+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:47:05.397538+0000","last_peered":"2024-04-23T01:47:05.397538+0000","last_clean":"2024-04-23T01:47:05.397538+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:47:05.397538+0000","last_undegraded":"2024-04-23T01:47:05.397538+0000","last_fullsized":"2024-04-23T01:47:05.397538+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.576933+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:47:05.576933+0000","last_peered":"2024-04-23T01:47:05.576933+0000","last_clean":"2024-04-23T01:47:05.576933+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:47:05.576933+0000","last_undegraded":"2024-04-23T01:47:05.576933+0000","last_fullsized":"2024-04-23T01:47:05.576933+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.590971+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:47:05.590971+0000","last_peered":"2024-04-23T01:47:05.590971+0000","last_clean":"2024-04-23T01:47:05.590971+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:47:05.590971+0000","last_undegraded":"2024-04-23T01:47:05.590971+0000","last_fullsized":"2024-04-23T01:47:05.590971+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.331332+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:47:05.331332+0000","last_peered":"2024-04-23T01:47:05.331332+0000","last_clean":"2024-04-23T01:47:05.331332+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:47:05.331332+0000","last_undegraded":"2024-04-23T01:47:05.331332+0000","last_fullsized":"2024-04-23T01:47:05.331332+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.602643+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:47:05.602643+0000","last_peered":"2024-04-23T01:47:05.602643+0000","last_clean":"2024-04-23T01:47:05.602643+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:47:05.602643+0000","last_undegraded":"2024-04-23T01:47:05.602643+0000","last_fullsized":"2024-04-23T01:47:05.602643+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.400601+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:47:05.400601+0000","last_peered":"2024-04-23T01:47:05.400601+0000","last_clean":"2024-04-23T01:47:05.400601+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:47:05.400601+0000","last_undegraded":"2024-04-23T01:47:05.400601+0000","last_fullsized":"2024-04-23T01:47:05.400601+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46504,"kb_used_data":1632,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94325336,"statfs":{"total":96636764160,"available":96589144064,"internally_reserved":0,"allocated":1671168,"data_stored":3399480,"data_compressed":170812,"data_compressed_allocated":1404928,"data_compressed_original":3301376,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58712,"kb_used_data":1360,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94313128,"statfs":{"total":96636764160,"available":96576643072,"internally_reserved":0,"allocated":1392640,"data_stored":2834256,"data_compressed":141625,"data_compressed_allocated":1122304,"data_compressed_original":2736128,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59056,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94312784,"statfs":{"total":96636764160,"available":96576290816,"internally_reserved":0,"allocated":1712128,"data_stored":3005744,"data_compressed":151011,"data_compressed_allocated":1462272,"data_compressed_original":2924544,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50124,"kb_used_data":1092,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94321716,"statfs":{"total":96636764160,"available":96585437184,"internally_reserved":0,"allocated":1118208,"data_stored":2285392,"data_compressed":113284,"data_compressed_allocated":847872,"data_compressed_original":2187264,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:48:50.696 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:48:50.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:50.803+0000 7fb921dd7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:50.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:50.810+0000 7fb921dd7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:50.994 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:48:50.994 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:48:51.007 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2024-04-23T01:48:50.311239+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":214396,"kb_used_data":5756,"kb_used_omap":0,"kb_used_meta":208576,"kb_avail":377272964,"statfs":{"total":386547056640,"available":386327515136,"internally_reserved":0,"allocated":5894144,"data_stored":11524872,"data_compressed":576732,"data_compressed_allocated":4837376,"data_compressed_original":11149312,"omap_allocated":0,"internal_metadata":213581824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-51552280,"num_objects":-788,"num_object_clones":0,"num_object_copies":-2364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-13998,"num_write_kb":-105129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.720569"},"pg_stats":[{"pgid":"1.0","version":"12'97","reported_seq":186,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.325282+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:47:05.325282+0000","last_peered":"2024-04-23T01:47:05.325282+0000","last_clean":"2024-04-23T01:47:05.325282+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:47:05.325282+0000","last_undegraded":"2024-04-23T01:47:05.325282+0000","last_fullsized":"2024-04-23T01:47:05.325282+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.410555+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:47:05.410555+0000","last_peered":"2024-04-23T01:47:05.410555+0000","last_clean":"2024-04-23T01:47:05.410555+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:47:05.410555+0000","last_undegraded":"2024-04-23T01:47:05.410555+0000","last_fullsized":"2024-04-23T01:47:05.410555+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.570797+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:47:05.570797+0000","last_peered":"2024-04-23T01:47:05.570797+0000","last_clean":"2024-04-23T01:47:05.570797+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:47:05.570797+0000","last_undegraded":"2024-04-23T01:47:05.570797+0000","last_fullsized":"2024-04-23T01:47:05.570797+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.397538+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:47:05.397538+0000","last_peered":"2024-04-23T01:47:05.397538+0000","last_clean":"2024-04-23T01:47:05.397538+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:47:05.397538+0000","last_undegraded":"2024-04-23T01:47:05.397538+0000","last_fullsized":"2024-04-23T01:47:05.397538+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.576933+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:47:05.576933+0000","last_peered":"2024-04-23T01:47:05.576933+0000","last_clean":"2024-04-23T01:47:05.576933+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:47:05.576933+0000","last_undegraded":"2024-04-23T01:47:05.576933+0000","last_fullsized":"2024-04-23T01:47:05.576933+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.590971+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:47:05.590971+0000","last_peered":"2024-04-23T01:47:05.590971+0000","last_clean":"2024-04-23T01:47:05.590971+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:47:05.590971+0000","last_undegraded":"2024-04-23T01:47:05.590971+0000","last_fullsized":"2024-04-23T01:47:05.590971+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.331332+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:47:05.331332+0000","last_peered":"2024-04-23T01:47:05.331332+0000","last_clean":"2024-04-23T01:47:05.331332+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:47:05.331332+0000","last_undegraded":"2024-04-23T01:47:05.331332+0000","last_fullsized":"2024-04-23T01:47:05.331332+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.602643+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:47:05.602643+0000","last_peered":"2024-04-23T01:47:05.602643+0000","last_clean":"2024-04-23T01:47:05.602643+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:47:05.602643+0000","last_undegraded":"2024-04-23T01:47:05.602643+0000","last_fullsized":"2024-04-23T01:47:05.602643+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-23T01:47:05.400601+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:47:05.400601+0000","last_peered":"2024-04-23T01:47:05.400601+0000","last_clean":"2024-04-23T01:47:05.400601+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:47:05.400601+0000","last_undegraded":"2024-04-23T01:47:05.400601+0000","last_fullsized":"2024-04-23T01:47:05.400601+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46504,"kb_used_data":1632,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94325336,"statfs":{"total":96636764160,"available":96589144064,"internally_reserved":0,"allocated":1671168,"data_stored":3399480,"data_compressed":170812,"data_compressed_allocated":1404928,"data_compressed_original":3301376,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58712,"kb_used_data":1360,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94313128,"statfs":{"total":96636764160,"available":96576643072,"internally_reserved":0,"allocated":1392640,"data_stored":2834256,"data_compressed":141625,"data_compressed_allocated":1122304,"data_compressed_original":2736128,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59056,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94312784,"statfs":{"total":96636764160,"available":96576290816,"internally_reserved":0,"allocated":1712128,"data_stored":3005744,"data_compressed":151011,"data_compressed_allocated":1462272,"data_compressed_original":2924544,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50124,"kb_used_data":1092,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94321716,"statfs":{"total":96636764160,"available":96585437184,"internally_reserved":0,"allocated":1118208,"data_stored":2285392,"data_compressed":113284,"data_compressed_allocated":847872,"data_compressed_original":2187264,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:48:51.007 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-23T01:48:51.008 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:48:51.008 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:48:51.008 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:48:51.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:51.149+0000 7fefc9606640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:51.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:51.158+0000 7fefc9606640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:51.174 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:48:51.174 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:48:51.175 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:48:51.175 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:48:51.175 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:48:51.175 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:48:51.175 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:48:51.175 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:48:51.175 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:48:51.175 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:48:51.175 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:48:51.175 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:48:51.177 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:48:51.177 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:48:51.178 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:48:51.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:51.283+0000 7f17a26ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:51.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:51.291+0000 7f17a26ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:51.306 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:48:51.307 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:48:51.307 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:48:51.307 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:48:51.307 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:48:51.307 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:48:51.307 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:48:51.308 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:48:51.308 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:48:51.308 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:48:51.308 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:48:51.308 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:48:51.310 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:48:51.311 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:48:51.311 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:48:51.312 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'unique_pool_2' created 2024-04-23T01:48:51.330 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_2 rados --yes-i-really-mean-it || true 2024-04-23T01:48:51.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:51.453+0000 7f5469e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:51.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:51.469+0000 7f5469e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:51.495 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:48:51.496 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:48:51.496 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:48:51.498 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:48:51.498 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:48:51.498 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:48:51.498 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:48:51.498 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:48:51.498 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:48:51.498 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:48:51.498 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:48:51.498 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:48:51.502 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:48:51.502 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:48:51.502 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:48:51.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:51.532+0000 7fdec106d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:51.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:51.552+0000 7fdec106d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:51.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:51.663+0000 7f57b346d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:51.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:51.676+0000 7f57b346d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:51.699 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:48:51.699 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:48:51.699 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:48:51.699 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:48:51.700 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:48:51.700 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:48:51.700 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:48:51.700 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:48:51.700 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:48:51.700 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:48:51.700 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:48:51.700 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:48:51.702 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:48:51.702 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:48:52.317 INFO:teuthology.orchestra.run.smithi090.stderr:enabled application 'rados' on pool 'unique_pool_2' 2024-04-23T01:48:52.768 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:48:52.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:52.870+0000 7f88a48ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:52.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:52.874+0000 7f88a48ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:53.338 DEBUG:teuthology.orchestra.run.smithi090:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 8192 --object-size 65536 -p unique_pool_2 bench 90 write' 2024-04-23T01:48:53.341 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-04-23T01:48:53.341 INFO:teuthology.orchestra.run:waiting for 3000 2024-04-23T01:48:53.402 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:hints = 1 2024-04-23T01:48:53.402 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Maintaining 16 concurrent writes of 8192 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-04-23T01:48:53.403 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Object prefix: benchmark_data_smithi090_37828 2024-04-23T01:48:53.403 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:48:53.403 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 0 0 0 0 0 0 - 0 2024-04-23T01:48:54.402 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 1 16 158 142 1.1093 1.10938 0.0782535 0.105798 2024-04-23T01:48:54.969 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:48:55.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:55.101+0000 7fd85006d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:55.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:55.109+0000 7fd85006d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:55.402 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 2 16 303 287 1.121 1.13281 0.194552 0.107101 2024-04-23T01:48:56.402 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 3 16 485 469 1.22126 1.42188 0.138725 0.101045 2024-04-23T01:48:56.703 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:48:56.704 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:48:56.704 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:48:56.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:56.822+0000 7f5f08bd7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:56.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:56.832+0000 7f5f08bd7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:57.055 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:48:57.055 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:48:57.072 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-04-23T01:48:55.592743+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":63,"acting":63,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":209840,"kb_used_data":1232,"kb_used_omap":0,"kb_used_meta":208576,"kb_avail":377277520,"statfs":{"total":386547056640,"available":386332180480,"internally_reserved":0,"allocated":1261568,"data_stored":2131499,"data_compressed":89859,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":213581824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.281504"},"pg_stats":[{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-23T01:48:51.429737+0000","last_change":"2024-04-23T01:48:51.429737+0000","last_active":"2024-04-23T01:48:51.310292+0000","last_peered":"2024-04-23T01:48:51.310292+0000","last_clean":"2024-04-23T01:48:51.310292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:48:51.429737+0000","last_undegraded":"2024-04-23T01:48:51.429737+0000","last_fullsized":"2024-04-23T01:48:51.429737+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:48:51.310292+0000","last_change":"2024-04-23T01:48:51.310292+0000","last_active":"2024-04-23T01:48:51.310292+0000","last_peered":"2024-04-23T01:48:51.310292+0000","last_clean":"2024-04-23T01:48:51.310292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:48:51.310292+0000","last_undegraded":"2024-04-23T01:48:51.310292+0000","last_fullsized":"2024-04-23T01:48:51.310292+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:48:51.310292+0000","last_change":"2024-04-23T01:48:51.310292+0000","last_active":"2024-04-23T01:48:51.310292+0000","last_peered":"2024-04-23T01:48:51.310292+0000","last_clean":"2024-04-23T01:48:51.310292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:48:51.310292+0000","last_undegraded":"2024-04-23T01:48:51.310292+0000","last_fullsized":"2024-04-23T01:48:51.310292+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-23T01:48:51.497820+0000","last_change":"2024-04-23T01:48:51.497820+0000","last_active":"2024-04-23T01:48:51.310292+0000","last_peered":"2024-04-23T01:48:51.310292+0000","last_clean":"2024-04-23T01:48:51.310292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:48:51.497820+0000","last_undegraded":"2024-04-23T01:48:51.497820+0000","last_fullsized":"2024-04-23T01:48:51.497820+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:48:51.310292+0000","last_change":"2024-04-23T01:48:51.310292+0000","last_active":"2024-04-23T01:48:51.310292+0000","last_peered":"2024-04-23T01:48:51.310292+0000","last_clean":"2024-04-23T01:48:51.310292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:48:51.310292+0000","last_undegraded":"2024-04-23T01:48:51.310292+0000","last_fullsized":"2024-04-23T01:48:51.310292+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-23T01:48:51.416067+0000","last_change":"2024-04-23T01:48:51.416067+0000","last_active":"2024-04-23T01:48:51.310292+0000","last_peered":"2024-04-23T01:48:51.310292+0000","last_clean":"2024-04-23T01:48:51.310292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:48:51.416067+0000","last_undegraded":"2024-04-23T01:48:51.416067+0000","last_fullsized":"2024-04-23T01:48:51.416067+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-23T01:48:51.540508+0000","last_change":"2024-04-23T01:48:51.540508+0000","last_active":"2024-04-23T01:48:51.310292+0000","last_peered":"2024-04-23T01:48:51.310292+0000","last_clean":"2024-04-23T01:48:51.310292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:48:51.540508+0000","last_undegraded":"2024-04-23T01:48:51.540508+0000","last_fullsized":"2024-04-23T01:48:51.540508+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:48:51.310292+0000","last_change":"2024-04-23T01:48:51.310292+0000","last_active":"2024-04-23T01:48:51.310292+0000","last_peered":"2024-04-23T01:48:51.310292+0000","last_clean":"2024-04-23T01:48:51.310292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:48:51.310292+0000","last_undegraded":"2024-04-23T01:48:51.310292+0000","last_fullsized":"2024-04-23T01:48:51.310292+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-23T01:48:51.369323+0000","last_change":"2024-04-23T01:48:51.369323+0000","last_active":"2024-04-23T01:48:51.310292+0000","last_peered":"2024-04-23T01:48:51.310292+0000","last_clean":"2024-04-23T01:48:51.310292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:48:51.369323+0000","last_undegraded":"2024-04-23T01:48:51.369323+0000","last_fullsized":"2024-04-23T01:48:51.369323+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":197,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-23T01:48:51.360628+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:48:51.360628+0000","last_peered":"2024-04-23T01:48:51.360628+0000","last_clean":"2024-04-23T01:48:51.360628+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:48:51.360628+0000","last_undegraded":"2024-04-23T01:48:51.360628+0000","last_fullsized":"2024-04-23T01:48:51.360628+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":64,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-23T01:48:51.445564+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:48:51.445564+0000","last_peered":"2024-04-23T01:48:51.445564+0000","last_clean":"2024-04-23T01:48:51.445564+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:48:51.445564+0000","last_undegraded":"2024-04-23T01:48:51.445564+0000","last_fullsized":"2024-04-23T01:48:51.445564+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-23T01:48:51.593379+0000","last_change":"2024-04-23T01:48:51.593379+0000","last_active":"2024-04-23T01:48:51.310292+0000","last_peered":"2024-04-23T01:48:51.310292+0000","last_clean":"2024-04-23T01:48:51.310292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:48:51.593379+0000","last_undegraded":"2024-04-23T01:48:51.593379+0000","last_fullsized":"2024-04-23T01:48:51.593379+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":62,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-23T01:48:51.414523+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:48:51.414523+0000","last_peered":"2024-04-23T01:48:51.414523+0000","last_clean":"2024-04-23T01:48:51.414523+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:48:51.414523+0000","last_undegraded":"2024-04-23T01:48:51.414523+0000","last_fullsized":"2024-04-23T01:48:51.414523+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-23T01:48:51.401879+0000","last_change":"2024-04-23T01:48:51.401879+0000","last_active":"2024-04-23T01:48:51.310292+0000","last_peered":"2024-04-23T01:48:51.310292+0000","last_clean":"2024-04-23T01:48:51.310292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:48:51.401879+0000","last_undegraded":"2024-04-23T01:48:51.401879+0000","last_fullsized":"2024-04-23T01:48:51.401879+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":64,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-23T01:48:51.390616+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:48:51.390616+0000","last_peered":"2024-04-23T01:48:51.390616+0000","last_clean":"2024-04-23T01:48:51.390616+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:48:51.390616+0000","last_undegraded":"2024-04-23T01:48:51.390616+0000","last_fullsized":"2024-04-23T01:48:51.390616+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-23T01:48:51.385973+0000","last_change":"2024-04-23T01:48:51.385973+0000","last_active":"2024-04-23T01:48:51.310292+0000","last_peered":"2024-04-23T01:48:51.310292+0000","last_clean":"2024-04-23T01:48:51.310292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:48:51.385973+0000","last_undegraded":"2024-04-23T01:48:51.385973+0000","last_fullsized":"2024-04-23T01:48:51.385973+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":62,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-23T01:48:51.427753+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:48:51.427753+0000","last_peered":"2024-04-23T01:48:51.427753+0000","last_clean":"2024-04-23T01:48:51.427753+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:48:51.427753+0000","last_undegraded":"2024-04-23T01:48:51.427753+0000","last_fullsized":"2024-04-23T01:48:51.427753+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-23T01:48:51.355823+0000","last_change":"2024-04-23T01:48:51.355823+0000","last_active":"2024-04-23T01:48:51.310292+0000","last_peered":"2024-04-23T01:48:51.310292+0000","last_clean":"2024-04-23T01:48:51.310292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:48:51.355823+0000","last_undegraded":"2024-04-23T01:48:51.355823+0000","last_fullsized":"2024-04-23T01:48:51.355823+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":68,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-23T01:48:51.349019+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:48:51.349019+0000","last_peered":"2024-04-23T01:48:51.349019+0000","last_clean":"2024-04-23T01:48:51.349019+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:48:51.349019+0000","last_undegraded":"2024-04-23T01:48:51.349019+0000","last_fullsized":"2024-04-23T01:48:51.349019+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-23T01:48:51.544751+0000","last_change":"2024-04-23T01:48:51.544751+0000","last_active":"2024-04-23T01:48:51.310292+0000","last_peered":"2024-04-23T01:48:51.310292+0000","last_clean":"2024-04-23T01:48:51.310292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:48:51.544751+0000","last_undegraded":"2024-04-23T01:48:51.544751+0000","last_fullsized":"2024-04-23T01:48:51.544751+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":64,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-23T01:48:51.406465+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:48:51.406465+0000","last_peered":"2024-04-23T01:48:51.406465+0000","last_clean":"2024-04-23T01:48:51.406465+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:48:51.406465+0000","last_undegraded":"2024-04-23T01:48:51.406465+0000","last_fullsized":"2024-04-23T01:48:51.406465+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-23T01:48:51.442474+0000","last_change":"2024-04-23T01:48:51.442474+0000","last_active":"2024-04-23T01:48:51.310292+0000","last_peered":"2024-04-23T01:48:51.310292+0000","last_clean":"2024-04-23T01:48:51.310292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:48:51.442474+0000","last_undegraded":"2024-04-23T01:48:51.442474+0000","last_fullsized":"2024-04-23T01:48:51.442474+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":68,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-23T01:48:51.332049+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:48:51.332049+0000","last_peered":"2024-04-23T01:48:51.332049+0000","last_clean":"2024-04-23T01:48:51.332049+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:48:51.332049+0000","last_undegraded":"2024-04-23T01:48:51.332049+0000","last_fullsized":"2024-04-23T01:48:51.332049+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-23T01:48:51.470529+0000","last_change":"2024-04-23T01:48:51.470529+0000","last_active":"2024-04-23T01:48:51.310292+0000","last_peered":"2024-04-23T01:48:51.310292+0000","last_clean":"2024-04-23T01:48:51.310292+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:48:51.470529+0000","last_undegraded":"2024-04-23T01:48:51.470529+0000","last_fullsized":"2024-04-23T01:48:51.470529+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":64,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-23T01:48:51.384424+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:48:51.384424+0000","last_peered":"2024-04-23T01:48:51.384424+0000","last_clean":"2024-04-23T01:48:51.384424+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:48:51.384424+0000","last_undegraded":"2024-04-23T01:48:51.384424+0000","last_fullsized":"2024-04-23T01:48:51.384424+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":36,"acting":36,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":45188,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94326652,"statfs":{"total":96636764160,"available":96590491648,"internally_reserved":0,"allocated":323584,"data_stored":678034,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":57688,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94314152,"statfs":{"total":96636764160,"available":96577691648,"internally_reserved":0,"allocated":344064,"data_stored":684520,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":57616,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94314224,"statfs":{"total":96636764160,"available":96577765376,"internally_reserved":0,"allocated":270336,"data_stored":90911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":49348,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94322492,"statfs":{"total":96636764160,"available":96586231808,"internally_reserved":0,"allocated":323584,"data_stored":678034,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:48:57.373 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:48:57.402 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 4 16 649 633 1.23623 1.28125 0.0638523 0.0996927 2024-04-23T01:48:57.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:57.510+0000 7fae0266d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:57.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:57.517+0000 7fae0266d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:58.403 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 5 15 798 783 1.22334 1.17188 0.0693473 0.10055 2024-04-23T01:48:59.403 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 6 16 976 960 1.2499 1.38281 0.0854291 0.0990699 2024-04-23T01:48:59.790 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:48:59.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:59.929+0000 7fdd522ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:48:59.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:48:59.936+0000 7fdd522ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:00.403 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 7 16 1119 1103 1.23093 1.11719 0.0474577 0.100295 2024-04-23T01:49:01.403 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 8 16 1274 1258 1.22841 1.21094 0.121124 0.101116 2024-04-23T01:49:02.074 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:49:02.075 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:49:02.075 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:49:02.075 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:49:02.141 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:49:02.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:02.205+0000 7f2eb366d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:02.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:02.219+0000 7f2eb366d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:02.277 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:49:02.277 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:49:02.277 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:49:02.277 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:49:02.277 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:49:02.277 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:49:02.277 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:49:02.277 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:49:02.277 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:49:02.277 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:49:02.277 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:49:02.277 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:49:02.280 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:49:02.281 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:49:02.281 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:49:02.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:02.284+0000 7fbf8f26d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:02.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:02.291+0000 7fbf8f26d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:02.403 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 9 15 1415 1400 1.21518 1.10938 0.12134 0.1021 2024-04-23T01:49:02.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:02.411+0000 7fee920ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:02.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:02.426+0000 7fee920ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:02.460 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:49:02.460 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:49:02.460 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:49:02.460 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:49:02.460 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:49:02.460 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:49:02.460 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:49:02.460 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:49:02.460 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:49:02.461 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:49:02.461 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:49:02.461 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:49:02.463 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:49:02.463 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:49:02.463 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:49:02.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:02.631+0000 7f581182d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:02.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:02.646+0000 7f581182d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:02.734 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:49:02.734 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:49:02.734 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:49:02.734 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:49:02.735 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:49:02.735 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:49:02.735 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:49:02.735 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:49:02.735 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:49:02.735 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:49:02.735 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:49:02.735 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:49:02.737 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:49:02.737 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:49:02.738 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:49:02.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:02.922+0000 7fc982154640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:02.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:02.935+0000 7fc982154640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:02.964 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:49:02.964 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:49:02.964 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:49:02.964 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:49:02.964 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:49:02.964 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:49:02.964 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:49:02.964 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:49:02.964 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:49:02.964 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:49:02.964 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:49:02.964 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:49:02.967 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:49:02.968 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:49:03.403 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 10 16 1564 1548 1.20928 1.15625 0.0680041 0.102897 2024-04-23T01:49:04.403 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 11 16 1730 1714 1.21723 1.29688 0.153834 0.102024 2024-04-23T01:49:04.645 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:49:04.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:04.773+0000 7fc92b66d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:04.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:04.779+0000 7fc92b66d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:05.403 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 12 16 1884 1868 1.21604 1.20312 0.108385 0.102348 2024-04-23T01:49:06.403 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 13 16 2044 2028 1.21865 1.25 0.190381 0.102062 2024-04-23T01:49:06.913 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:49:07.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:07.047+0000 7fa68ff5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:07.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:07.054+0000 7fa68ff5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:07.404 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 14 16 2187 2171 1.21139 1.11719 0.111538 0.102835 2024-04-23T01:49:07.969 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:49:07.969 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:49:07.970 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:49:08.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:08.094+0000 7fdde5f1d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:08.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:08.104+0000 7fdde5f1d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:08.404 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 15 16 2341 2325 1.21084 1.20312 0.219077 0.102707 2024-04-23T01:49:08.444 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:49:08.445 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:49:08.461 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-04-23T01:49:07.594966+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":17449504,"num_objects":260,"num_object_clones":0,"num_object_copies":780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":2195,"num_write_kb":18358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2139,"ondisk_log_size":2139,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":225004,"kb_used_data":24716,"kb_used_omap":0,"kb_used_meta":200256,"kb_avail":377262356,"statfs":{"total":386547056640,"available":386316652544,"internally_reserved":0,"allocated":25309184,"data_stored":49982888,"data_compressed":2554311,"data_compressed_allocated":24018944,"data_compressed_original":49512448,"omap_allocated":0,"internal_metadata":205062144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16859136,"num_objects":258,"num_object_clones":0,"num_object_copies":774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2058,"num_write_kb":16464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002222"},"pg_stats":[{"pgid":"5.f","version":"36'79","reported_seq":92,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:04.486038+0000","last_change":"2024-04-23T01:48:52.530638+0000","last_active":"2024-04-23T01:49:04.486038+0000","last_peered":"2024-04-23T01:49:04.486038+0000","last_clean":"2024-04-23T01:49:04.486038+0000","last_became_active":"2024-04-23T01:48:52.520759+0000","last_became_peered":"2024-04-23T01:48:52.520759+0000","last_unstale":"2024-04-23T01:49:04.486038+0000","last_undegraded":"2024-04-23T01:49:04.486038+0000","last_fullsized":"2024-04-23T01:49:04.486038+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"36'111","reported_seq":124,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:06.290620+0000","last_change":"2024-04-23T01:48:52.532479+0000","last_active":"2024-04-23T01:49:06.290620+0000","last_peered":"2024-04-23T01:49:06.290620+0000","last_clean":"2024-04-23T01:49:06.290620+0000","last_became_active":"2024-04-23T01:48:52.525595+0000","last_became_peered":"2024-04-23T01:48:52.525595+0000","last_unstale":"2024-04-23T01:49:06.290620+0000","last_undegraded":"2024-04-23T01:49:06.290620+0000","last_fullsized":"2024-04-23T01:49:06.290620+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"36'191","reported_seq":204,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:06.472104+0000","last_change":"2024-04-23T01:48:52.571199+0000","last_active":"2024-04-23T01:49:06.472104+0000","last_peered":"2024-04-23T01:49:06.472104+0000","last_clean":"2024-04-23T01:49:06.472104+0000","last_became_active":"2024-04-23T01:48:52.566277+0000","last_became_peered":"2024-04-23T01:48:52.566277+0000","last_unstale":"2024-04-23T01:49:06.472104+0000","last_undegraded":"2024-04-23T01:49:06.472104+0000","last_fullsized":"2024-04-23T01:49:06.472104+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1572864,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":192,"num_write_kb":1536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"36'95","reported_seq":108,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:05.230887+0000","last_change":"2024-04-23T01:48:52.497141+0000","last_active":"2024-04-23T01:49:05.230887+0000","last_peered":"2024-04-23T01:49:05.230887+0000","last_clean":"2024-04-23T01:49:05.230887+0000","last_became_active":"2024-04-23T01:48:52.489329+0000","last_became_peered":"2024-04-23T01:48:52.489329+0000","last_unstale":"2024-04-23T01:49:05.230887+0000","last_undegraded":"2024-04-23T01:49:05.230887+0000","last_fullsized":"2024-04-23T01:49:05.230887+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":786432,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96,"num_write_kb":768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"36'151","reported_seq":164,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:06.355157+0000","last_change":"2024-04-23T01:48:52.534975+0000","last_active":"2024-04-23T01:49:06.355157+0000","last_peered":"2024-04-23T01:49:06.355157+0000","last_clean":"2024-04-23T01:49:06.355157+0000","last_became_active":"2024-04-23T01:48:52.530664+0000","last_became_peered":"2024-04-23T01:48:52.530664+0000","last_unstale":"2024-04-23T01:49:06.355157+0000","last_undegraded":"2024-04-23T01:49:06.355157+0000","last_fullsized":"2024-04-23T01:49:06.355157+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"36'151","reported_seq":164,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:05.295695+0000","last_change":"2024-04-23T01:48:52.451606+0000","last_active":"2024-04-23T01:49:05.295695+0000","last_peered":"2024-04-23T01:49:05.295695+0000","last_clean":"2024-04-23T01:49:05.295695+0000","last_became_active":"2024-04-23T01:48:52.439020+0000","last_became_peered":"2024-04-23T01:48:52.439020+0000","last_unstale":"2024-04-23T01:49:05.295695+0000","last_undegraded":"2024-04-23T01:49:05.295695+0000","last_fullsized":"2024-04-23T01:49:05.295695+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"36'71","reported_seq":84,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:05.363199+0000","last_change":"2024-04-23T01:48:52.481437+0000","last_active":"2024-04-23T01:49:05.363199+0000","last_peered":"2024-04-23T01:49:05.363199+0000","last_clean":"2024-04-23T01:49:05.363199+0000","last_became_active":"2024-04-23T01:48:52.473748+0000","last_became_peered":"2024-04-23T01:48:52.473748+0000","last_unstale":"2024-04-23T01:49:05.363199+0000","last_undegraded":"2024-04-23T01:49:05.363199+0000","last_fullsized":"2024-04-23T01:49:05.363199+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"36'111","reported_seq":124,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:06.376720+0000","last_change":"2024-04-23T01:48:52.568521+0000","last_active":"2024-04-23T01:49:06.376720+0000","last_peered":"2024-04-23T01:49:06.376720+0000","last_clean":"2024-04-23T01:49:06.376720+0000","last_became_active":"2024-04-23T01:48:52.562008+0000","last_became_peered":"2024-04-23T01:48:52.562008+0000","last_unstale":"2024-04-23T01:49:06.376720+0000","last_undegraded":"2024-04-23T01:49:06.376720+0000","last_fullsized":"2024-04-23T01:49:06.376720+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"36'127","reported_seq":140,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:06.441933+0000","last_change":"2024-04-23T01:48:52.491272+0000","last_active":"2024-04-23T01:49:06.441933+0000","last_peered":"2024-04-23T01:49:06.441933+0000","last_clean":"2024-04-23T01:49:06.441933+0000","last_became_active":"2024-04-23T01:48:52.482878+0000","last_became_peered":"2024-04-23T01:48:52.482878+0000","last_unstale":"2024-04-23T01:49:06.441933+0000","last_undegraded":"2024-04-23T01:49:06.441933+0000","last_fullsized":"2024-04-23T01:49:06.441933+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":203,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.336526+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:48:53.336526+0000","last_peered":"2024-04-23T01:48:53.336526+0000","last_clean":"2024-04-23T01:48:53.336526+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:48:53.336526+0000","last_undegraded":"2024-04-23T01:48:53.336526+0000","last_fullsized":"2024-04-23T01:48:53.336526+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.441413+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:48:53.441413+0000","last_peered":"2024-04-23T01:48:53.441413+0000","last_clean":"2024-04-23T01:48:53.441413+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:48:53.441413+0000","last_undegraded":"2024-04-23T01:48:53.441413+0000","last_fullsized":"2024-04-23T01:48:53.441413+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"36'153","reported_seq":166,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:06.591847+0000","last_change":"2024-04-23T01:48:52.550172+0000","last_active":"2024-04-23T01:49:06.591847+0000","last_peered":"2024-04-23T01:49:06.591847+0000","last_clean":"2024-04-23T01:49:06.591847+0000","last_became_active":"2024-04-23T01:48:52.541104+0000","last_became_peered":"2024-04-23T01:48:52.541104+0000","last_unstale":"2024-04-23T01:49:06.591847+0000","last_undegraded":"2024-04-23T01:49:06.591847+0000","last_fullsized":"2024-04-23T01:49:06.591847+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1261568,"num_objects":20,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":154,"num_write_kb":1232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.571998+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:48:53.571998+0000","last_peered":"2024-04-23T01:48:53.571998+0000","last_clean":"2024-04-23T01:48:53.571998+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:48:53.571998+0000","last_undegraded":"2024-04-23T01:48:53.571998+0000","last_fullsized":"2024-04-23T01:48:53.571998+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"36'103","reported_seq":116,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:05.797165+0000","last_change":"2024-04-23T01:48:52.526429+0000","last_active":"2024-04-23T01:49:05.797165+0000","last_peered":"2024-04-23T01:49:05.797165+0000","last_clean":"2024-04-23T01:49:05.797165+0000","last_became_active":"2024-04-23T01:48:52.514040+0000","last_became_peered":"2024-04-23T01:48:52.514040+0000","last_unstale":"2024-04-23T01:49:05.797165+0000","last_undegraded":"2024-04-23T01:49:05.797165+0000","last_fullsized":"2024-04-23T01:49:05.797165+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.432658+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:48:53.432658+0000","last_peered":"2024-04-23T01:48:53.432658+0000","last_clean":"2024-04-23T01:48:53.432658+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:48:53.432658+0000","last_undegraded":"2024-04-23T01:48:53.432658+0000","last_fullsized":"2024-04-23T01:48:53.432658+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"36'183","reported_seq":196,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:05.853899+0000","last_change":"2024-04-23T01:48:52.469625+0000","last_active":"2024-04-23T01:49:05.853899+0000","last_peered":"2024-04-23T01:49:05.853899+0000","last_clean":"2024-04-23T01:49:05.853899+0000","last_became_active":"2024-04-23T01:48:52.456212+0000","last_became_peered":"2024-04-23T01:48:52.456212+0000","last_unstale":"2024-04-23T01:49:05.853899+0000","last_undegraded":"2024-04-23T01:49:05.853899+0000","last_fullsized":"2024-04-23T01:49:05.853899+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":68,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.605161+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:48:53.605161+0000","last_peered":"2024-04-23T01:48:53.605161+0000","last_clean":"2024-04-23T01:48:53.605161+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:48:53.605161+0000","last_undegraded":"2024-04-23T01:48:53.605161+0000","last_fullsized":"2024-04-23T01:48:53.605161+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"36'135","reported_seq":148,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:06.178391+0000","last_change":"2024-04-23T01:48:52.505215+0000","last_active":"2024-04-23T01:49:06.178391+0000","last_peered":"2024-04-23T01:49:06.178391+0000","last_clean":"2024-04-23T01:49:06.178391+0000","last_became_active":"2024-04-23T01:48:52.497519+0000","last_became_peered":"2024-04-23T01:48:52.497519+0000","last_unstale":"2024-04-23T01:49:06.178391+0000","last_undegraded":"2024-04-23T01:49:06.178391+0000","last_fullsized":"2024-04-23T01:49:06.178391+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":136,"num_write_kb":1088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.438431+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:48:53.438431+0000","last_peered":"2024-04-23T01:48:53.438431+0000","last_clean":"2024-04-23T01:48:53.438431+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:48:53.438431+0000","last_undegraded":"2024-04-23T01:48:53.438431+0000","last_fullsized":"2024-04-23T01:48:53.438431+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"36'151","reported_seq":164,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:05.551891+0000","last_change":"2024-04-23T01:48:52.501817+0000","last_active":"2024-04-23T01:49:05.551891+0000","last_peered":"2024-04-23T01:49:05.551891+0000","last_clean":"2024-04-23T01:49:05.551891+0000","last_became_active":"2024-04-23T01:48:52.496130+0000","last_became_peered":"2024-04-23T01:48:52.496130+0000","last_unstale":"2024-04-23T01:49:05.551891+0000","last_undegraded":"2024-04-23T01:49:05.551891+0000","last_fullsized":"2024-04-23T01:49:05.551891+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.346665+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:48:53.346665+0000","last_peered":"2024-04-23T01:48:53.346665+0000","last_clean":"2024-04-23T01:48:53.346665+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:48:53.346665+0000","last_undegraded":"2024-04-23T01:48:53.346665+0000","last_fullsized":"2024-04-23T01:48:53.346665+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"36'119","reported_seq":132,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:06.079741+0000","last_change":"2024-04-23T01:48:52.473323+0000","last_active":"2024-04-23T01:49:06.079741+0000","last_peered":"2024-04-23T01:49:06.079741+0000","last_clean":"2024-04-23T01:49:06.079741+0000","last_became_active":"2024-04-23T01:48:52.466602+0000","last_became_peered":"2024-04-23T01:48:52.466602+0000","last_unstale":"2024-04-23T01:49:06.079741+0000","last_undegraded":"2024-04-23T01:49:06.079741+0000","last_fullsized":"2024-04-23T01:49:06.079741+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.439757+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:48:53.439757+0000","last_peered":"2024-04-23T01:48:53.439757+0000","last_clean":"2024-04-23T01:48:53.439757+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:48:53.439757+0000","last_undegraded":"2024-04-23T01:48:53.439757+0000","last_fullsized":"2024-04-23T01:48:53.439757+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"36'111","reported_seq":124,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:06.514983+0000","last_change":"2024-04-23T01:48:52.464660+0000","last_active":"2024-04-23T01:49:06.514983+0000","last_peered":"2024-04-23T01:49:06.514983+0000","last_clean":"2024-04-23T01:49:06.514983+0000","last_became_active":"2024-04-23T01:48:52.455507+0000","last_became_peered":"2024-04-23T01:48:52.455507+0000","last_unstale":"2024-04-23T01:49:06.514983+0000","last_undegraded":"2024-04-23T01:49:06.514983+0000","last_fullsized":"2024-04-23T01:49:06.514983+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.430351+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:48:53.430351+0000","last_peered":"2024-04-23T01:48:53.430351+0000","last_clean":"2024-04-23T01:48:53.430351+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:48:53.430351+0000","last_undegraded":"2024-04-23T01:48:53.430351+0000","last_fullsized":"2024-04-23T01:48:53.430351+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":16859136,"num_objects":258,"num_object_clones":0,"num_object_copies":774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2058,"num_write_kb":16464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":25292800,"data_stored":50585600,"data_compressed":2608695,"data_compressed_allocated":25292800,"data_compressed_original":50585600,"omap_allocated":0,"internal_metadata":0},"log_size":2042,"ondisk_log_size":2042,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59216,"kb_used_data":6024,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312624,"statfs":{"total":96636764160,"available":96576126976,"internally_reserved":0,"allocated":6168576,"data_stored":12295154,"data_compressed":628101,"data_compressed_allocated":5840896,"data_compressed_original":12173312,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":56016,"kb_used_data":6984,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94315824,"statfs":{"total":96636764160,"available":96579403776,"internally_reserved":0,"allocated":7151616,"data_stored":14261234,"data_compressed":729491,"data_compressed_allocated":6823936,"data_compressed_original":14139392,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50036,"kb_used_data":5164,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94321804,"statfs":{"total":96636764160,"available":96585527296,"internally_reserved":0,"allocated":5287936,"data_stored":10066386,"data_compressed":513662,"data_compressed_allocated":4980736,"data_compressed_original":9961472,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59736,"kb_used_data":6544,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312104,"statfs":{"total":96636764160,"available":96575594496,"internally_reserved":0,"allocated":6701056,"data_stored":13360114,"data_compressed":683057,"data_compressed_allocated":6373376,"data_compressed_original":13238272,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6729728,"data_stored":13459456,"data_compressed":694135,"data_compressed_allocated":6729728,"data_compressed_original":13459456,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5320704,"data_stored":10641408,"data_compressed":548771,"data_compressed_allocated":5320704,"data_compressed_original":10641408,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7180288,"data_stored":14360576,"data_compressed":740569,"data_compressed_allocated":7180288,"data_compressed_original":14360576,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6062080,"data_stored":12124160,"data_compressed":625220,"data_compressed_allocated":6062080,"data_compressed_original":12124160,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:49:09.267 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:49:09.404 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 16 16 2483 2467 1.20449 1.10938 0.0817648 0.1034 2024-04-23T01:49:09.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:09.424+0000 7f545d39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:09.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:09.431+0000 7f545d39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:10.404 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 17 16 2620 2604 1.19659 1.07031 0.179878 0.104085 2024-04-23T01:49:11.404 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 18 16 2789 2773 1.20346 1.32031 0.0501136 0.103583 2024-04-23T01:49:11.691 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:49:11.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:11.820+0000 7f923575d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:11.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:11.827+0000 7f923575d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:12.404 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 19 16 2951 2935 1.20672 1.26562 0.194037 0.103197 2024-04-23T01:49:13.404 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:49:13.405445+0000 min lat: 0.014864 max lat: 0.31614 avg lat: 0.10323 2024-04-23T01:49:13.404 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:49:13.404 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 20 16 3109 3093 1.2081 1.23438 0.123878 0.10323 2024-04-23T01:49:13.464 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:49:13.464 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:49:13.464 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:49:13.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:13.589+0000 7f2b1262d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:13.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:13.599+0000 7f2b1262d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:13.859 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:49:13.860 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:49:13.876 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-04-23T01:49:13.595963+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":23609888,"num_objects":355,"num_object_clones":0,"num_object_copies":1065,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":2947,"num_write_kb":24374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2891,"ondisk_log_size":2891,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":242212,"kb_used_data":33604,"kb_used_omap":0,"kb_used_meta":208576,"kb_avail":377245148,"statfs":{"total":386547056640,"available":386299031552,"internally_reserved":0,"allocated":34410496,"data_stored":68185512,"data_compressed":3494217,"data_compressed_allocated":33120256,"data_compressed_original":67715072,"omap_allocated":0,"internal_metadata":213581824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18874368,"num_objects":288,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2304,"num_write_kb":18432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002134"},"pg_stats":[{"pgid":"5.f","version":"36'127","reported_seq":140,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:11.532452+0000","last_change":"2024-04-23T01:48:52.530638+0000","last_active":"2024-04-23T01:49:11.532452+0000","last_peered":"2024-04-23T01:49:11.532452+0000","last_clean":"2024-04-23T01:49:11.532452+0000","last_became_active":"2024-04-23T01:48:52.520759+0000","last_became_peered":"2024-04-23T01:48:52.520759+0000","last_unstale":"2024-04-23T01:49:11.532452+0000","last_undegraded":"2024-04-23T01:49:11.532452+0000","last_fullsized":"2024-04-23T01:49:11.532452+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"36'151","reported_seq":164,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:10.124513+0000","last_change":"2024-04-23T01:48:52.532479+0000","last_active":"2024-04-23T01:49:10.124513+0000","last_peered":"2024-04-23T01:49:10.124513+0000","last_clean":"2024-04-23T01:49:10.124513+0000","last_became_active":"2024-04-23T01:48:52.525595+0000","last_became_peered":"2024-04-23T01:48:52.525595+0000","last_unstale":"2024-04-23T01:49:10.124513+0000","last_undegraded":"2024-04-23T01:49:10.124513+0000","last_fullsized":"2024-04-23T01:49:10.124513+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"36'226","reported_seq":239,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:11.567219+0000","last_change":"2024-04-23T01:48:52.571199+0000","last_active":"2024-04-23T01:49:11.567219+0000","last_peered":"2024-04-23T01:49:11.567219+0000","last_clean":"2024-04-23T01:49:11.567219+0000","last_became_active":"2024-04-23T01:48:52.566277+0000","last_became_peered":"2024-04-23T01:48:52.566277+0000","last_unstale":"2024-04-23T01:49:11.567219+0000","last_undegraded":"2024-04-23T01:49:11.567219+0000","last_fullsized":"2024-04-23T01:49:11.567219+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1859584,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":227,"num_write_kb":1816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"36'151","reported_seq":164,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:11.357117+0000","last_change":"2024-04-23T01:48:52.497141+0000","last_active":"2024-04-23T01:49:11.357117+0000","last_peered":"2024-04-23T01:49:11.357117+0000","last_clean":"2024-04-23T01:49:11.357117+0000","last_became_active":"2024-04-23T01:48:52.489329+0000","last_became_peered":"2024-04-23T01:48:52.489329+0000","last_unstale":"2024-04-23T01:49:11.357117+0000","last_undegraded":"2024-04-23T01:49:11.357117+0000","last_fullsized":"2024-04-23T01:49:11.357117+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"36'183","reported_seq":196,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:10.755263+0000","last_change":"2024-04-23T01:48:52.534975+0000","last_active":"2024-04-23T01:49:10.755263+0000","last_peered":"2024-04-23T01:49:10.755263+0000","last_clean":"2024-04-23T01:49:10.755263+0000","last_became_active":"2024-04-23T01:48:52.530664+0000","last_became_peered":"2024-04-23T01:48:52.530664+0000","last_unstale":"2024-04-23T01:49:10.755263+0000","last_undegraded":"2024-04-23T01:49:10.755263+0000","last_fullsized":"2024-04-23T01:49:10.755263+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"36'167","reported_seq":180,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:10.390741+0000","last_change":"2024-04-23T01:48:52.451606+0000","last_active":"2024-04-23T01:49:10.390741+0000","last_peered":"2024-04-23T01:49:10.390741+0000","last_clean":"2024-04-23T01:49:10.390741+0000","last_became_active":"2024-04-23T01:48:52.439020+0000","last_became_peered":"2024-04-23T01:48:52.439020+0000","last_unstale":"2024-04-23T01:49:10.390741+0000","last_undegraded":"2024-04-23T01:49:10.390741+0000","last_fullsized":"2024-04-23T01:49:10.390741+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":1344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"36'130","reported_seq":143,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:11.602670+0000","last_change":"2024-04-23T01:48:52.481437+0000","last_active":"2024-04-23T01:49:11.602670+0000","last_peered":"2024-04-23T01:49:11.602670+0000","last_clean":"2024-04-23T01:49:11.602670+0000","last_became_active":"2024-04-23T01:48:52.473748+0000","last_became_peered":"2024-04-23T01:48:52.473748+0000","last_unstale":"2024-04-23T01:49:11.602670+0000","last_undegraded":"2024-04-23T01:49:11.602670+0000","last_fullsized":"2024-04-23T01:49:11.602670+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1073152,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":131,"num_write_kb":1048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"36'135","reported_seq":148,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:10.969500+0000","last_change":"2024-04-23T01:48:52.568521+0000","last_active":"2024-04-23T01:49:10.969500+0000","last_peered":"2024-04-23T01:49:10.969500+0000","last_clean":"2024-04-23T01:49:10.969500+0000","last_became_active":"2024-04-23T01:48:52.562008+0000","last_became_peered":"2024-04-23T01:48:52.562008+0000","last_unstale":"2024-04-23T01:49:10.969500+0000","last_undegraded":"2024-04-23T01:49:10.969500+0000","last_fullsized":"2024-04-23T01:49:10.969500+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":136,"num_write_kb":1088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"36'183","reported_seq":196,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:11.021514+0000","last_change":"2024-04-23T01:48:52.491272+0000","last_active":"2024-04-23T01:49:11.021514+0000","last_peered":"2024-04-23T01:49:11.021514+0000","last_clean":"2024-04-23T01:49:11.021514+0000","last_became_active":"2024-04-23T01:48:52.482878+0000","last_became_peered":"2024-04-23T01:48:52.482878+0000","last_unstale":"2024-04-23T01:49:11.021514+0000","last_undegraded":"2024-04-23T01:49:11.021514+0000","last_fullsized":"2024-04-23T01:49:11.021514+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":203,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.336526+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:48:53.336526+0000","last_peered":"2024-04-23T01:48:53.336526+0000","last_clean":"2024-04-23T01:48:53.336526+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:48:53.336526+0000","last_undegraded":"2024-04-23T01:48:53.336526+0000","last_fullsized":"2024-04-23T01:48:53.336526+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.441413+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:48:53.441413+0000","last_peered":"2024-04-23T01:48:53.441413+0000","last_clean":"2024-04-23T01:48:53.441413+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:48:53.441413+0000","last_undegraded":"2024-04-23T01:48:53.441413+0000","last_fullsized":"2024-04-23T01:48:53.441413+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"36'191","reported_seq":204,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:11.068999+0000","last_change":"2024-04-23T01:48:52.550172+0000","last_active":"2024-04-23T01:49:11.068999+0000","last_peered":"2024-04-23T01:49:11.068999+0000","last_clean":"2024-04-23T01:49:11.068999+0000","last_became_active":"2024-04-23T01:48:52.541104+0000","last_became_peered":"2024-04-23T01:48:52.541104+0000","last_unstale":"2024-04-23T01:49:11.068999+0000","last_undegraded":"2024-04-23T01:49:11.068999+0000","last_fullsized":"2024-04-23T01:49:11.068999+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1572864,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":192,"num_write_kb":1536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.571998+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:48:53.571998+0000","last_peered":"2024-04-23T01:48:53.571998+0000","last_clean":"2024-04-23T01:48:53.571998+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:48:53.571998+0000","last_undegraded":"2024-04-23T01:48:53.571998+0000","last_fullsized":"2024-04-23T01:48:53.571998+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"36'159","reported_seq":172,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:11.114961+0000","last_change":"2024-04-23T01:48:52.526429+0000","last_active":"2024-04-23T01:49:11.114961+0000","last_peered":"2024-04-23T01:49:11.114961+0000","last_clean":"2024-04-23T01:49:11.114961+0000","last_became_active":"2024-04-23T01:48:52.514040+0000","last_became_peered":"2024-04-23T01:48:52.514040+0000","last_unstale":"2024-04-23T01:49:11.114961+0000","last_undegraded":"2024-04-23T01:49:11.114961+0000","last_fullsized":"2024-04-23T01:49:11.114961+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310720,"num_objects":20,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":160,"num_write_kb":1280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.432658+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:48:53.432658+0000","last_peered":"2024-04-23T01:48:53.432658+0000","last_clean":"2024-04-23T01:48:53.432658+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:48:53.432658+0000","last_undegraded":"2024-04-23T01:48:53.432658+0000","last_fullsized":"2024-04-23T01:48:53.432658+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"36'271","reported_seq":284,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:09.974148+0000","last_change":"2024-04-23T01:48:52.469625+0000","last_active":"2024-04-23T01:49:09.974148+0000","last_peered":"2024-04-23T01:49:09.974148+0000","last_clean":"2024-04-23T01:49:09.974148+0000","last_became_active":"2024-04-23T01:48:52.456212+0000","last_became_peered":"2024-04-23T01:48:52.456212+0000","last_unstale":"2024-04-23T01:49:09.974148+0000","last_undegraded":"2024-04-23T01:49:09.974148+0000","last_fullsized":"2024-04-23T01:49:09.974148+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":272,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":68,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.605161+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:48:53.605161+0000","last_peered":"2024-04-23T01:48:53.605161+0000","last_clean":"2024-04-23T01:48:53.605161+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:48:53.605161+0000","last_undegraded":"2024-04-23T01:48:53.605161+0000","last_fullsized":"2024-04-23T01:48:53.605161+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"36'183","reported_seq":196,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:11.296149+0000","last_change":"2024-04-23T01:48:52.505215+0000","last_active":"2024-04-23T01:49:11.296149+0000","last_peered":"2024-04-23T01:49:11.296149+0000","last_clean":"2024-04-23T01:49:11.296149+0000","last_became_active":"2024-04-23T01:48:52.497519+0000","last_became_peered":"2024-04-23T01:48:52.497519+0000","last_unstale":"2024-04-23T01:49:11.296149+0000","last_undegraded":"2024-04-23T01:49:11.296149+0000","last_fullsized":"2024-04-23T01:49:11.296149+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.438431+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:48:53.438431+0000","last_peered":"2024-04-23T01:48:53.438431+0000","last_clean":"2024-04-23T01:48:53.438431+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:48:53.438431+0000","last_undegraded":"2024-04-23T01:48:53.438431+0000","last_fullsized":"2024-04-23T01:48:53.438431+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"36'207","reported_seq":220,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:11.581615+0000","last_change":"2024-04-23T01:48:52.501817+0000","last_active":"2024-04-23T01:49:11.581615+0000","last_peered":"2024-04-23T01:49:11.581615+0000","last_clean":"2024-04-23T01:49:11.581615+0000","last_became_active":"2024-04-23T01:48:52.496130+0000","last_became_peered":"2024-04-23T01:48:52.496130+0000","last_unstale":"2024-04-23T01:49:11.581615+0000","last_undegraded":"2024-04-23T01:49:11.581615+0000","last_fullsized":"2024-04-23T01:49:11.581615+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703936,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":208,"num_write_kb":1664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.346665+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:48:53.346665+0000","last_peered":"2024-04-23T01:48:53.346665+0000","last_clean":"2024-04-23T01:48:53.346665+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:48:53.346665+0000","last_undegraded":"2024-04-23T01:48:53.346665+0000","last_fullsized":"2024-04-23T01:48:53.346665+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"36'163","reported_seq":176,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:11.566374+0000","last_change":"2024-04-23T01:48:52.473323+0000","last_active":"2024-04-23T01:49:11.566374+0000","last_peered":"2024-04-23T01:49:11.566374+0000","last_clean":"2024-04-23T01:49:11.566374+0000","last_became_active":"2024-04-23T01:48:52.466602+0000","last_became_peered":"2024-04-23T01:48:52.466602+0000","last_unstale":"2024-04-23T01:49:11.566374+0000","last_undegraded":"2024-04-23T01:49:11.566374+0000","last_fullsized":"2024-04-23T01:49:11.566374+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1343488,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":164,"num_write_kb":1312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.439757+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:48:53.439757+0000","last_peered":"2024-04-23T01:48:53.439757+0000","last_clean":"2024-04-23T01:48:53.439757+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:48:53.439757+0000","last_undegraded":"2024-04-23T01:48:53.439757+0000","last_fullsized":"2024-04-23T01:48:53.439757+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"36'167","reported_seq":180,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:11.430972+0000","last_change":"2024-04-23T01:48:52.464660+0000","last_active":"2024-04-23T01:49:11.430972+0000","last_peered":"2024-04-23T01:49:11.430972+0000","last_clean":"2024-04-23T01:49:11.430972+0000","last_became_active":"2024-04-23T01:48:52.455507+0000","last_became_peered":"2024-04-23T01:48:52.455507+0000","last_unstale":"2024-04-23T01:49:11.430972+0000","last_undegraded":"2024-04-23T01:49:11.430972+0000","last_fullsized":"2024-04-23T01:49:11.430972+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":1344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.430351+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:48:53.430351+0000","last_peered":"2024-04-23T01:48:53.430351+0000","last_clean":"2024-04-23T01:48:53.430351+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:48:53.430351+0000","last_undegraded":"2024-04-23T01:48:53.430351+0000","last_fullsized":"2024-04-23T01:48:53.430351+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":23019520,"num_objects":353,"num_object_clones":0,"num_object_copies":1059,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2810,"num_write_kb":22480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":34512896,"data_stored":69025792,"data_compressed":3560868,"data_compressed_allocated":34512896,"data_compressed_original":69025792,"omap_allocated":0,"internal_metadata":0},"log_size":2794,"ondisk_log_size":2794,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":61200,"kb_used_data":8008,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94310640,"statfs":{"total":96636764160,"available":96574095360,"internally_reserved":0,"allocated":8200192,"data_stored":16358386,"data_compressed":837909,"data_compressed_allocated":7872512,"data_compressed_original":16236544,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58560,"kb_used_data":9528,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94313280,"statfs":{"total":96636764160,"available":96576798720,"internally_reserved":0,"allocated":9756672,"data_stored":19471346,"data_compressed":998519,"data_compressed_allocated":9428992,"data_compressed_original":19349504,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":56336,"kb_used_data":7304,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94315504,"statfs":{"total":96636764160,"available":96579076096,"internally_reserved":0,"allocated":7479296,"data_stored":14449106,"data_compressed":739967,"data_compressed_allocated":7172096,"data_compressed_original":14344192,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":66116,"kb_used_data":8764,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94305724,"statfs":{"total":96636764160,"available":96569061376,"internally_reserved":0,"allocated":8974336,"data_stored":17906674,"data_compressed":917822,"data_compressed_allocated":8646656,"data_compressed_original":17784832,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9019392,"data_stored":18038784,"data_compressed":930592,"data_compressed_allocated":9019392,"data_compressed_original":18038784,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7417856,"data_stored":14835712,"data_compressed":765347,"data_compressed_allocated":7417856,"data_compressed_original":14835712,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":9895936,"data_stored":19791872,"data_compressed":1021018,"data_compressed_allocated":9895936,"data_compressed_original":19791872,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8179712,"data_stored":16359424,"data_compressed":843911,"data_compressed_allocated":8179712,"data_compressed_original":16359424,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:49:14.012 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:49:14.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:14.127+0000 7f301de2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:14.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:14.133+0000 7f301de2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:14.404 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 21 16 3260 3244 1.20674 1.17969 0.148239 0.103215 2024-04-23T01:49:15.404 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 22 16 3423 3407 1.20977 1.27344 0.0769123 0.103077 2024-04-23T01:49:16.310 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:49:16.404 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 23 16 3564 3548 1.20506 1.10156 0.151545 0.103333 2024-04-23T01:49:16.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:16.426+0000 7f8b38a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:16.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:16.433+0000 7f8b38a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:17.404 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 24 16 3704 3688 1.20042 1.09375 0.152793 0.103779 2024-04-23T01:49:18.405 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 25 16 3867 3851 1.20334 1.27344 0.171531 0.103543 2024-04-23T01:49:18.689 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:49:18.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:18.822+0000 7f932b6ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:18.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:18.830+0000 7f932b6ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:18.878 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:49:18.878 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:49:18.878 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:49:18.878 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:49:19.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:19.002+0000 7f7ec7f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:19.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:19.014+0000 7f7ec7f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:19.051 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:49:19.051 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:49:19.051 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:49:19.051 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:49:19.051 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:49:19.051 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:49:19.051 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:49:19.051 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:49:19.051 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:49:19.051 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:49:19.051 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:49:19.051 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:49:19.054 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:49:19.054 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:49:19.055 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:49:19.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:19.234+0000 7ff11f66d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:19.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:19.244+0000 7ff11f66d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:19.262 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:49:19.262 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:49:19.262 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:49:19.263 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:49:19.263 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:49:19.263 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:49:19.263 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:49:19.263 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:49:19.263 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:49:19.263 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:49:19.263 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:49:19.263 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:49:19.266 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:49:19.266 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:49:19.266 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:49:19.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:19.387+0000 7f5c38606640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:19.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:19.400+0000 7f5c38606640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:19.404 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 26 16 4002 3986 1.19762 1.05469 0.05217 0.103989 2024-04-23T01:49:19.432 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:49:19.432 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:49:19.432 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:49:19.432 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:49:19.432 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:49:19.432 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:49:19.432 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:49:19.432 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:49:19.432 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:49:19.432 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:49:19.433 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:49:19.433 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:49:19.435 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:49:19.436 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:49:19.436 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:49:19.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:19.562+0000 7ffaf34ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:19.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:19.576+0000 7ffaf34ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:19.598 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:49:19.598 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:49:19.598 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:49:19.598 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:49:19.598 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:49:19.598 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:49:19.598 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:49:19.599 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:49:19.599 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:49:19.599 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:49:19.599 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:49:19.599 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:49:19.602 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:49:19.602 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:49:20.405 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 27 16 4172 4156 1.20245 1.32812 0.136675 0.103725 2024-04-23T01:49:21.036 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:49:21.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:21.168+0000 7fc23da6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:21.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:21.175+0000 7fc23da6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:21.405 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 28 16 4316 4300 1.19968 1.125 0.0717853 0.103944 2024-04-23T01:49:22.405 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 29 16 4464 4448 1.19818 1.15625 0.148763 0.10409 2024-04-23T01:49:23.327 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:49:23.405 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 30 16 4620 4604 1.19886 1.21875 0.112905 0.104103 2024-04-23T01:49:23.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:23.460+0000 7f1aafd5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:23.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:23.468+0000 7f1aafd5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:24.405 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 31 16 4767 4751 1.19723 1.14844 0.0977349 0.104159 2024-04-23T01:49:24.603 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:49:24.604 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:49:24.604 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:49:24.604 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:49:24.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:24.717+0000 7f1eb439d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:24.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:24.728+0000 7f1eb439d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:24.816 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:49:24.817 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:49:24.817 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:49:24.817 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:49:24.817 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:49:24.817 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:49:24.817 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:49:24.817 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:49:24.817 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:49:24.817 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:49:24.817 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:49:24.817 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:49:24.819 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:49:24.820 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:49:24.820 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:49:24.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:24.951+0000 7f78e999d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:24.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:24.961+0000 7f78e999d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:24.979 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:49:24.979 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:49:24.979 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:49:24.980 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:49:24.980 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:49:24.980 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:49:24.980 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:49:24.980 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:49:24.980 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:49:24.980 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:49:24.980 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:49:24.980 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:49:24.983 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:49:24.983 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:49:24.984 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:49:25.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:25.100+0000 7fcc4c6ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:25.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:25.110+0000 7fcc4c6ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:25.275 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:49:25.276 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:49:25.276 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:49:25.276 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:49:25.276 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:49:25.276 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:49:25.276 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:49:25.276 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:49:25.276 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:49:25.276 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:49:25.276 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:49:25.276 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:49:25.278 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:49:25.279 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:49:25.279 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:49:25.405 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 32 16 4900 4884 1.19228 1.03906 0.118469 0.104611 2024-04-23T01:49:25.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:25.410+0000 7f53fe46d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:25.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:25.421+0000 7f53fe46d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:25.444 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:49:25.444 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:49:25.444 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:49:25.444 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:49:25.444 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:49:25.444 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:49:25.444 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:49:25.444 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:49:25.445 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:49:25.445 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:49:25.445 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:49:25.445 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:49:25.447 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:49:25.448 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:49:25.648 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:49:25.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:25.774+0000 7f6b3ea6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:25.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:25.781+0000 7f6b3ea6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:26.405 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 33 16 5058 5042 1.19356 1.23438 0.125288 0.104553 2024-04-23T01:49:27.405 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 34 16 5196 5180 1.19016 1.07812 0.246116 0.104745 2024-04-23T01:49:28.004 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:49:28.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:28.194+0000 7f1f3595d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:28.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:28.202+0000 7f1f3595d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:28.405 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 35 16 5330 5314 1.18606 1.04688 0.0978585 0.105172 2024-04-23T01:49:29.405 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 36 16 5490 5474 1.18783 1.25 0.0796289 0.105066 2024-04-23T01:49:30.406 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 37 16 5645 5629 1.18846 1.21094 0.173394 0.104967 2024-04-23T01:49:30.433 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:49:30.449 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:49:30.449 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:49:30.450 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:49:30.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:30.557+0000 7fcd15b9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:30.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:30.566+0000 7fcd15b9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:30.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:30.599+0000 7f916a19d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:30.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:30.610+0000 7f916a19d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:30.857 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:49:30.858 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:49:30.875 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-04-23T01:49:29.599035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":42189344,"num_objects":638,"num_object_clones":0,"num_object_copies":1914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":5215,"num_write_kb":42518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5159,"ondisk_log_size":5159,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":265356,"kb_used_data":60904,"kb_used_omap":0,"kb_used_meta":204416,"kb_avail":377222004,"statfs":{"total":386547056640,"available":386275332096,"internally_reserved":0,"allocated":62365696,"data_stored":124095912,"data_compressed":6381192,"data_compressed_allocated":61075456,"data_compressed_original":123625472,"omap_allocated":0,"internal_metadata":209321984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12271616,"num_objects":186,"num_object_clones":0,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1498,"num_write_kb":11984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002353"},"pg_stats":[{"pgid":"5.f","version":"36'264","reported_seq":277,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:25.508853+0000","last_change":"2024-04-23T01:48:52.530638+0000","last_active":"2024-04-23T01:49:25.508853+0000","last_peered":"2024-04-23T01:49:25.508853+0000","last_clean":"2024-04-23T01:49:25.508853+0000","last_became_active":"2024-04-23T01:48:52.520759+0000","last_became_peered":"2024-04-23T01:48:52.520759+0000","last_unstale":"2024-04-23T01:49:25.508853+0000","last_undegraded":"2024-04-23T01:49:25.508853+0000","last_fullsized":"2024-04-23T01:49:25.508853+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2170880,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":265,"num_write_kb":2120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"36'303","reported_seq":316,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:25.839752+0000","last_change":"2024-04-23T01:48:52.532479+0000","last_active":"2024-04-23T01:49:25.839752+0000","last_peered":"2024-04-23T01:49:25.839752+0000","last_clean":"2024-04-23T01:49:25.839752+0000","last_became_active":"2024-04-23T01:48:52.525595+0000","last_became_peered":"2024-04-23T01:48:52.525595+0000","last_unstale":"2024-04-23T01:49:25.839752+0000","last_undegraded":"2024-04-23T01:49:25.839752+0000","last_fullsized":"2024-04-23T01:49:25.839752+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490368,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"36'447","reported_seq":460,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:24.891611+0000","last_change":"2024-04-23T01:48:52.571199+0000","last_active":"2024-04-23T01:49:24.891611+0000","last_peered":"2024-04-23T01:49:24.891611+0000","last_clean":"2024-04-23T01:49:24.891611+0000","last_became_active":"2024-04-23T01:48:52.566277+0000","last_became_peered":"2024-04-23T01:48:52.566277+0000","last_unstale":"2024-04-23T01:49:24.891611+0000","last_undegraded":"2024-04-23T01:49:24.891611+0000","last_fullsized":"2024-04-23T01:49:24.891611+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":448,"num_write_kb":3584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"36'263","reported_seq":276,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:26.546459+0000","last_change":"2024-04-23T01:48:52.497141+0000","last_active":"2024-04-23T01:49:26.546459+0000","last_peered":"2024-04-23T01:49:26.546459+0000","last_clean":"2024-04-23T01:49:26.546459+0000","last_became_active":"2024-04-23T01:48:52.489329+0000","last_became_peered":"2024-04-23T01:48:52.489329+0000","last_unstale":"2024-04-23T01:49:26.546459+0000","last_undegraded":"2024-04-23T01:49:26.546459+0000","last_fullsized":"2024-04-23T01:49:26.546459+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162688,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":264,"num_write_kb":2112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"36'303","reported_seq":316,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:26.491637+0000","last_change":"2024-04-23T01:48:52.534975+0000","last_active":"2024-04-23T01:49:26.491637+0000","last_peered":"2024-04-23T01:49:26.491637+0000","last_clean":"2024-04-23T01:49:26.491637+0000","last_became_active":"2024-04-23T01:48:52.530664+0000","last_became_peered":"2024-04-23T01:48:52.530664+0000","last_unstale":"2024-04-23T01:49:26.491637+0000","last_undegraded":"2024-04-23T01:49:26.491637+0000","last_fullsized":"2024-04-23T01:49:26.491637+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490368,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"36'319","reported_seq":332,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:26.312664+0000","last_change":"2024-04-23T01:48:52.451606+0000","last_active":"2024-04-23T01:49:26.312664+0000","last_peered":"2024-04-23T01:49:26.312664+0000","last_clean":"2024-04-23T01:49:26.312664+0000","last_became_active":"2024-04-23T01:48:52.439020+0000","last_became_peered":"2024-04-23T01:48:52.439020+0000","last_unstale":"2024-04-23T01:49:26.312664+0000","last_undegraded":"2024-04-23T01:49:26.312664+0000","last_fullsized":"2024-04-23T01:49:26.312664+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":320,"num_write_kb":2560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"36'231","reported_seq":244,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:26.218577+0000","last_change":"2024-04-23T01:48:52.481437+0000","last_active":"2024-04-23T01:49:26.218577+0000","last_peered":"2024-04-23T01:49:26.218577+0000","last_clean":"2024-04-23T01:49:26.218577+0000","last_became_active":"2024-04-23T01:48:52.473748+0000","last_became_peered":"2024-04-23T01:48:52.473748+0000","last_unstale":"2024-04-23T01:49:26.218577+0000","last_undegraded":"2024-04-23T01:49:26.218577+0000","last_fullsized":"2024-04-23T01:49:26.218577+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1900544,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":232,"num_write_kb":1856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"36'295","reported_seq":308,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:26.345773+0000","last_change":"2024-04-23T01:48:52.568521+0000","last_active":"2024-04-23T01:49:26.345773+0000","last_peered":"2024-04-23T01:49:26.345773+0000","last_clean":"2024-04-23T01:49:26.345773+0000","last_became_active":"2024-04-23T01:48:52.562008+0000","last_became_peered":"2024-04-23T01:48:52.562008+0000","last_unstale":"2024-04-23T01:49:26.345773+0000","last_undegraded":"2024-04-23T01:49:26.345773+0000","last_fullsized":"2024-04-23T01:49:26.345773+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2424832,"num_objects":37,"num_object_clones":0,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":296,"num_write_kb":2368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"36'319","reported_seq":332,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:26.415155+0000","last_change":"2024-04-23T01:48:52.491272+0000","last_active":"2024-04-23T01:49:26.415155+0000","last_peered":"2024-04-23T01:49:26.415155+0000","last_clean":"2024-04-23T01:49:26.415155+0000","last_became_active":"2024-04-23T01:48:52.482878+0000","last_became_peered":"2024-04-23T01:48:52.482878+0000","last_unstale":"2024-04-23T01:49:26.415155+0000","last_undegraded":"2024-04-23T01:49:26.415155+0000","last_fullsized":"2024-04-23T01:49:26.415155+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":320,"num_write_kb":2560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":203,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.336526+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:48:53.336526+0000","last_peered":"2024-04-23T01:48:53.336526+0000","last_clean":"2024-04-23T01:48:53.336526+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:48:53.336526+0000","last_undegraded":"2024-04-23T01:48:53.336526+0000","last_fullsized":"2024-04-23T01:48:53.336526+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.441413+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:48:53.441413+0000","last_peered":"2024-04-23T01:48:53.441413+0000","last_clean":"2024-04-23T01:48:53.441413+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:48:53.441413+0000","last_undegraded":"2024-04-23T01:48:53.441413+0000","last_fullsized":"2024-04-23T01:48:53.441413+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"36'343","reported_seq":356,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:26.440826+0000","last_change":"2024-04-23T01:48:52.550172+0000","last_active":"2024-04-23T01:49:26.440826+0000","last_peered":"2024-04-23T01:49:26.440826+0000","last_clean":"2024-04-23T01:49:26.440826+0000","last_became_active":"2024-04-23T01:48:52.541104+0000","last_became_peered":"2024-04-23T01:48:52.541104+0000","last_unstale":"2024-04-23T01:49:26.440826+0000","last_undegraded":"2024-04-23T01:49:26.440826+0000","last_fullsized":"2024-04-23T01:49:26.440826+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":344,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.571998+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:48:53.571998+0000","last_peered":"2024-04-23T01:48:53.571998+0000","last_clean":"2024-04-23T01:48:53.571998+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:48:53.571998+0000","last_undegraded":"2024-04-23T01:48:53.571998+0000","last_fullsized":"2024-04-23T01:48:53.571998+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"36'308","reported_seq":321,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:26.570988+0000","last_change":"2024-04-23T01:48:52.526429+0000","last_active":"2024-04-23T01:49:26.570988+0000","last_peered":"2024-04-23T01:49:26.570988+0000","last_clean":"2024-04-23T01:49:26.570988+0000","last_became_active":"2024-04-23T01:48:52.514040+0000","last_became_peered":"2024-04-23T01:48:52.514040+0000","last_unstale":"2024-04-23T01:49:26.570988+0000","last_undegraded":"2024-04-23T01:49:26.570988+0000","last_fullsized":"2024-04-23T01:49:26.570988+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2531328,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":309,"num_write_kb":2472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.432658+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:48:53.432658+0000","last_peered":"2024-04-23T01:48:53.432658+0000","last_clean":"2024-04-23T01:48:53.432658+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:48:53.432658+0000","last_undegraded":"2024-04-23T01:48:53.432658+0000","last_fullsized":"2024-04-23T01:48:53.432658+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"36'471","reported_seq":484,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:24.128045+0000","last_change":"2024-04-23T01:48:52.469625+0000","last_active":"2024-04-23T01:49:24.128045+0000","last_peered":"2024-04-23T01:49:24.128045+0000","last_clean":"2024-04-23T01:49:24.128045+0000","last_became_active":"2024-04-23T01:48:52.456212+0000","last_became_peered":"2024-04-23T01:48:52.456212+0000","last_unstale":"2024-04-23T01:49:24.128045+0000","last_undegraded":"2024-04-23T01:49:24.128045+0000","last_fullsized":"2024-04-23T01:49:24.128045+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3866624,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":472,"num_write_kb":3776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":68,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.605161+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:48:53.605161+0000","last_peered":"2024-04-23T01:48:53.605161+0000","last_clean":"2024-04-23T01:48:53.605161+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:48:53.605161+0000","last_undegraded":"2024-04-23T01:48:53.605161+0000","last_fullsized":"2024-04-23T01:48:53.605161+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"36'303","reported_seq":316,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:26.141645+0000","last_change":"2024-04-23T01:48:52.505215+0000","last_active":"2024-04-23T01:49:26.141645+0000","last_peered":"2024-04-23T01:49:26.141645+0000","last_clean":"2024-04-23T01:49:26.141645+0000","last_became_active":"2024-04-23T01:48:52.497519+0000","last_became_peered":"2024-04-23T01:48:52.497519+0000","last_unstale":"2024-04-23T01:49:26.141645+0000","last_undegraded":"2024-04-23T01:49:26.141645+0000","last_fullsized":"2024-04-23T01:49:26.141645+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490368,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.438431+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:48:53.438431+0000","last_peered":"2024-04-23T01:48:53.438431+0000","last_clean":"2024-04-23T01:48:53.438431+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:48:53.438431+0000","last_undegraded":"2024-04-23T01:48:53.438431+0000","last_fullsized":"2024-04-23T01:48:53.438431+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"36'351","reported_seq":364,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:24.412895+0000","last_change":"2024-04-23T01:48:52.501817+0000","last_active":"2024-04-23T01:49:24.412895+0000","last_peered":"2024-04-23T01:49:24.412895+0000","last_clean":"2024-04-23T01:49:24.412895+0000","last_became_active":"2024-04-23T01:48:52.496130+0000","last_became_peered":"2024-04-23T01:48:52.496130+0000","last_unstale":"2024-04-23T01:49:24.412895+0000","last_undegraded":"2024-04-23T01:49:24.412895+0000","last_fullsized":"2024-04-23T01:49:24.412895+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883584,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":352,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.346665+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:48:53.346665+0000","last_peered":"2024-04-23T01:48:53.346665+0000","last_clean":"2024-04-23T01:48:53.346665+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:48:53.346665+0000","last_undegraded":"2024-04-23T01:48:53.346665+0000","last_fullsized":"2024-04-23T01:48:53.346665+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"36'279","reported_seq":292,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:26.577293+0000","last_change":"2024-04-23T01:48:52.473323+0000","last_active":"2024-04-23T01:49:26.577293+0000","last_peered":"2024-04-23T01:49:26.577293+0000","last_clean":"2024-04-23T01:49:26.577293+0000","last_became_active":"2024-04-23T01:48:52.466602+0000","last_became_peered":"2024-04-23T01:48:52.466602+0000","last_unstale":"2024-04-23T01:49:26.577293+0000","last_undegraded":"2024-04-23T01:49:26.577293+0000","last_fullsized":"2024-04-23T01:49:26.577293+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2293760,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":280,"num_write_kb":2240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.439757+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:48:53.439757+0000","last_peered":"2024-04-23T01:48:53.439757+0000","last_clean":"2024-04-23T01:48:53.439757+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:48:53.439757+0000","last_undegraded":"2024-04-23T01:48:53.439757+0000","last_fullsized":"2024-04-23T01:48:53.439757+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"36'263","reported_seq":276,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:26.064908+0000","last_change":"2024-04-23T01:48:52.464660+0000","last_active":"2024-04-23T01:49:26.064908+0000","last_peered":"2024-04-23T01:49:26.064908+0000","last_clean":"2024-04-23T01:49:26.064908+0000","last_became_active":"2024-04-23T01:48:52.455507+0000","last_became_peered":"2024-04-23T01:48:52.455507+0000","last_unstale":"2024-04-23T01:49:26.064908+0000","last_undegraded":"2024-04-23T01:49:26.064908+0000","last_fullsized":"2024-04-23T01:49:26.064908+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162688,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":264,"num_write_kb":2112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.430351+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:48:53.430351+0000","last_peered":"2024-04-23T01:48:53.430351+0000","last_clean":"2024-04-23T01:48:53.430351+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:48:53.430351+0000","last_undegraded":"2024-04-23T01:48:53.430351+0000","last_fullsized":"2024-04-23T01:48:53.430351+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":41598976,"num_objects":636,"num_object_clones":0,"num_object_copies":1908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5078,"num_write_kb":40624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":62390272,"data_stored":124780544,"data_compressed":6439806,"data_compressed_allocated":62390272,"data_compressed_original":124780544,"omap_allocated":0,"internal_metadata":0},"log_size":5062,"ondisk_log_size":5062,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59428,"kb_used_data":14556,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94312412,"statfs":{"total":96636764160,"available":96575909888,"internally_reserved":0,"allocated":14905344,"data_stored":29768690,"data_compressed":1530360,"data_compressed_allocated":14577664,"data_compressed_original":29646848,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":74944,"kb_used_data":17588,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94296896,"statfs":{"total":96636764160,"available":96560021504,"internally_reserved":0,"allocated":18010112,"data_stored":35978226,"data_compressed":1850864,"data_compressed_allocated":17682432,"data_compressed_original":35856384,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65808,"kb_used_data":12616,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94306032,"statfs":{"total":96636764160,"available":96569376768,"internally_reserved":0,"allocated":12918784,"data_stored":25328082,"data_compressed":1301711,"data_compressed_allocated":12611584,"data_compressed_original":25223168,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65176,"kb_used_data":16144,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94306664,"statfs":{"total":96636764160,"available":96570023936,"internally_reserved":0,"allocated":16531456,"data_stored":33020914,"data_compressed":1698257,"data_compressed_allocated":16203776,"data_compressed_original":32899072,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16535552,"data_stored":33071104,"data_compressed":1706797,"data_compressed_allocated":16535552,"data_compressed_original":33071104,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12910592,"data_stored":25821184,"data_compressed":1332590,"data_compressed_allocated":12910592,"data_compressed_original":25821184,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18112512,"data_stored":36225024,"data_compressed":1869556,"data_compressed_allocated":18112512,"data_compressed_original":36225024,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":14831616,"data_stored":29663232,"data_compressed":1530863,"data_compressed_allocated":14831616,"data_compressed_original":29663232,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:49:31.406 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 38 16 5787 5771 1.18637 1.10938 0.0804969 0.105226 2024-04-23T01:49:32.406 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 39 16 5944 5928 1.1874 1.22656 0.0478231 0.105092 2024-04-23T01:49:32.708 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:49:32.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:32.830+0000 7faa1995d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:32.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:32.837+0000 7faa1995d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:33.406 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:49:33.407125+0000 min lat: 0.014864 max lat: 0.319669 avg lat: 0.105151 2024-04-23T01:49:33.406 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:49:33.406 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 40 16 6093 6077 1.18681 1.16406 0.118248 0.105151 2024-04-23T01:49:34.406 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 41 16 6249 6233 1.18759 1.21875 0.0481753 0.105064 2024-04-23T01:49:35.092 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:49:35.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:35.231+0000 7ff0a866d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:35.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:35.238+0000 7ff0a866d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:35.406 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 42 16 6391 6375 1.18573 1.10938 0.171485 0.105261 2024-04-23T01:49:35.876 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:49:35.877 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:49:35.877 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:49:36.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:36.019+0000 7f84bbe6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:36.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:36.028+0000 7f84bbe6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:36.339 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:49:36.340 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:49:36.356 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2024-04-23T01:49:35.600177+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":48161312,"num_objects":730,"num_object_clones":0,"num_object_copies":2190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":5944,"num_write_kb":48350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5888,"ondisk_log_size":5888,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":261680,"kb_used_data":69712,"kb_used_omap":0,"kb_used_meta":191936,"kb_avail":377225680,"statfs":{"total":386547056640,"available":386279096320,"internally_reserved":0,"allocated":71385088,"data_stored":142134696,"data_compressed":7312638,"data_compressed_allocated":70094848,"data_compressed_original":141664256,"omap_allocated":0,"internal_metadata":196542464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12083200,"num_objects":185,"num_object_clones":0,"num_object_copies":555,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1475,"num_write_kb":11800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002373"},"pg_stats":[{"pgid":"5.f","version":"36'351","reported_seq":364,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:30.742604+0000","last_change":"2024-04-23T01:48:52.530638+0000","last_active":"2024-04-23T01:49:30.742604+0000","last_peered":"2024-04-23T01:49:30.742604+0000","last_clean":"2024-04-23T01:49:30.742604+0000","last_became_active":"2024-04-23T01:48:52.520759+0000","last_became_peered":"2024-04-23T01:48:52.520759+0000","last_unstale":"2024-04-23T01:49:30.742604+0000","last_undegraded":"2024-04-23T01:49:30.742604+0000","last_fullsized":"2024-04-23T01:49:30.742604+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883584,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":352,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"36'343","reported_seq":356,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:31.566738+0000","last_change":"2024-04-23T01:48:52.532479+0000","last_active":"2024-04-23T01:49:31.566738+0000","last_peered":"2024-04-23T01:49:31.566738+0000","last_clean":"2024-04-23T01:49:31.566738+0000","last_became_active":"2024-04-23T01:48:52.525595+0000","last_became_peered":"2024-04-23T01:48:52.525595+0000","last_unstale":"2024-04-23T01:49:31.566738+0000","last_undegraded":"2024-04-23T01:49:31.566738+0000","last_fullsized":"2024-04-23T01:49:31.566738+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":344,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"36'479","reported_seq":492,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:31.416561+0000","last_change":"2024-04-23T01:48:52.571199+0000","last_active":"2024-04-23T01:49:31.416561+0000","last_peered":"2024-04-23T01:49:31.416561+0000","last_clean":"2024-04-23T01:49:31.416561+0000","last_became_active":"2024-04-23T01:48:52.566277+0000","last_became_peered":"2024-04-23T01:48:52.566277+0000","last_unstale":"2024-04-23T01:49:31.416561+0000","last_undegraded":"2024-04-23T01:49:31.416561+0000","last_fullsized":"2024-04-23T01:49:31.416561+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":480,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"36'303","reported_seq":316,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:31.348282+0000","last_change":"2024-04-23T01:48:52.497141+0000","last_active":"2024-04-23T01:49:31.348282+0000","last_peered":"2024-04-23T01:49:31.348282+0000","last_clean":"2024-04-23T01:49:31.348282+0000","last_became_active":"2024-04-23T01:48:52.489329+0000","last_became_peered":"2024-04-23T01:48:52.489329+0000","last_unstale":"2024-04-23T01:49:31.348282+0000","last_undegraded":"2024-04-23T01:49:31.348282+0000","last_fullsized":"2024-04-23T01:49:31.348282+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490368,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"36'327","reported_seq":340,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:31.485704+0000","last_change":"2024-04-23T01:48:52.534975+0000","last_active":"2024-04-23T01:49:31.485704+0000","last_peered":"2024-04-23T01:49:31.485704+0000","last_clean":"2024-04-23T01:49:31.485704+0000","last_became_active":"2024-04-23T01:48:52.530664+0000","last_became_peered":"2024-04-23T01:48:52.530664+0000","last_unstale":"2024-04-23T01:49:31.485704+0000","last_undegraded":"2024-04-23T01:49:31.485704+0000","last_fullsized":"2024-04-23T01:49:31.485704+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2686976,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":328,"num_write_kb":2624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"36'375","reported_seq":388,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:31.577876+0000","last_change":"2024-04-23T01:48:52.451606+0000","last_active":"2024-04-23T01:49:31.577876+0000","last_peered":"2024-04-23T01:49:31.577876+0000","last_clean":"2024-04-23T01:49:31.577876+0000","last_became_active":"2024-04-23T01:48:52.439020+0000","last_became_peered":"2024-04-23T01:48:52.439020+0000","last_unstale":"2024-04-23T01:49:31.577876+0000","last_undegraded":"2024-04-23T01:49:31.577876+0000","last_fullsized":"2024-04-23T01:49:31.577876+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":3008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"36'279","reported_seq":292,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:29.801852+0000","last_change":"2024-04-23T01:48:52.481437+0000","last_active":"2024-04-23T01:49:29.801852+0000","last_peered":"2024-04-23T01:49:29.801852+0000","last_clean":"2024-04-23T01:49:29.801852+0000","last_became_active":"2024-04-23T01:48:52.473748+0000","last_became_peered":"2024-04-23T01:48:52.473748+0000","last_unstale":"2024-04-23T01:49:29.801852+0000","last_undegraded":"2024-04-23T01:49:29.801852+0000","last_fullsized":"2024-04-23T01:49:29.801852+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2293760,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":280,"num_write_kb":2240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"36'352","reported_seq":365,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:30.776208+0000","last_change":"2024-04-23T01:48:52.568521+0000","last_active":"2024-04-23T01:49:30.776208+0000","last_peered":"2024-04-23T01:49:30.776208+0000","last_clean":"2024-04-23T01:49:30.776208+0000","last_became_active":"2024-04-23T01:48:52.562008+0000","last_became_peered":"2024-04-23T01:48:52.562008+0000","last_unstale":"2024-04-23T01:49:30.776208+0000","last_undegraded":"2024-04-23T01:49:30.776208+0000","last_fullsized":"2024-04-23T01:49:30.776208+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2891776,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":353,"num_write_kb":2824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"36'351","reported_seq":364,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:31.043863+0000","last_change":"2024-04-23T01:48:52.491272+0000","last_active":"2024-04-23T01:49:31.043863+0000","last_peered":"2024-04-23T01:49:31.043863+0000","last_clean":"2024-04-23T01:49:31.043863+0000","last_became_active":"2024-04-23T01:48:52.482878+0000","last_became_peered":"2024-04-23T01:48:52.482878+0000","last_unstale":"2024-04-23T01:49:31.043863+0000","last_undegraded":"2024-04-23T01:49:31.043863+0000","last_fullsized":"2024-04-23T01:49:31.043863+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883584,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":352,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":203,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.336526+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:48:53.336526+0000","last_peered":"2024-04-23T01:48:53.336526+0000","last_clean":"2024-04-23T01:48:53.336526+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:48:53.336526+0000","last_undegraded":"2024-04-23T01:48:53.336526+0000","last_fullsized":"2024-04-23T01:48:53.336526+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.441413+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:48:53.441413+0000","last_peered":"2024-04-23T01:48:53.441413+0000","last_clean":"2024-04-23T01:48:53.441413+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:48:53.441413+0000","last_undegraded":"2024-04-23T01:48:53.441413+0000","last_fullsized":"2024-04-23T01:48:53.441413+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"36'399","reported_seq":412,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:31.307318+0000","last_change":"2024-04-23T01:48:52.550172+0000","last_active":"2024-04-23T01:49:31.307318+0000","last_peered":"2024-04-23T01:49:31.307318+0000","last_clean":"2024-04-23T01:49:31.307318+0000","last_became_active":"2024-04-23T01:48:52.541104+0000","last_became_peered":"2024-04-23T01:48:52.541104+0000","last_unstale":"2024-04-23T01:49:31.307318+0000","last_undegraded":"2024-04-23T01:49:31.307318+0000","last_fullsized":"2024-04-23T01:49:31.307318+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3276800,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":400,"num_write_kb":3200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.571998+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:48:53.571998+0000","last_peered":"2024-04-23T01:48:53.571998+0000","last_clean":"2024-04-23T01:48:53.571998+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:48:53.571998+0000","last_undegraded":"2024-04-23T01:48:53.571998+0000","last_fullsized":"2024-04-23T01:48:53.571998+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"36'344","reported_seq":357,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:29.690714+0000","last_change":"2024-04-23T01:48:52.526429+0000","last_active":"2024-04-23T01:49:29.690714+0000","last_peered":"2024-04-23T01:49:29.690714+0000","last_clean":"2024-04-23T01:49:29.690714+0000","last_became_active":"2024-04-23T01:48:52.514040+0000","last_became_peered":"2024-04-23T01:48:52.514040+0000","last_unstale":"2024-04-23T01:49:29.690714+0000","last_undegraded":"2024-04-23T01:49:29.690714+0000","last_fullsized":"2024-04-23T01:49:29.690714+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2826240,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":345,"num_write_kb":2760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.432658+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:48:53.432658+0000","last_peered":"2024-04-23T01:48:53.432658+0000","last_clean":"2024-04-23T01:48:53.432658+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:48:53.432658+0000","last_undegraded":"2024-04-23T01:48:53.432658+0000","last_fullsized":"2024-04-23T01:48:53.432658+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"36'511","reported_seq":524,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:31.454633+0000","last_change":"2024-04-23T01:48:52.469625+0000","last_active":"2024-04-23T01:49:31.454633+0000","last_peered":"2024-04-23T01:49:31.454633+0000","last_clean":"2024-04-23T01:49:31.454633+0000","last_became_active":"2024-04-23T01:48:52.456212+0000","last_became_peered":"2024-04-23T01:48:52.456212+0000","last_unstale":"2024-04-23T01:49:31.454633+0000","last_undegraded":"2024-04-23T01:49:31.454633+0000","last_fullsized":"2024-04-23T01:49:31.454633+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":512,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":68,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.605161+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:48:53.605161+0000","last_peered":"2024-04-23T01:48:53.605161+0000","last_clean":"2024-04-23T01:48:53.605161+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:48:53.605161+0000","last_undegraded":"2024-04-23T01:48:53.605161+0000","last_fullsized":"2024-04-23T01:48:53.605161+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"36'311","reported_seq":324,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:29.734585+0000","last_change":"2024-04-23T01:48:52.505215+0000","last_active":"2024-04-23T01:49:29.734585+0000","last_peered":"2024-04-23T01:49:29.734585+0000","last_clean":"2024-04-23T01:49:29.734585+0000","last_became_active":"2024-04-23T01:48:52.497519+0000","last_became_peered":"2024-04-23T01:48:52.497519+0000","last_unstale":"2024-04-23T01:49:29.734585+0000","last_undegraded":"2024-04-23T01:49:29.734585+0000","last_fullsized":"2024-04-23T01:49:29.734585+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2555904,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":2496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.438431+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:48:53.438431+0000","last_peered":"2024-04-23T01:48:53.438431+0000","last_clean":"2024-04-23T01:48:53.438431+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:48:53.438431+0000","last_undegraded":"2024-04-23T01:48:53.438431+0000","last_fullsized":"2024-04-23T01:48:53.438431+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"36'415","reported_seq":428,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:31.240332+0000","last_change":"2024-04-23T01:48:52.501817+0000","last_active":"2024-04-23T01:49:31.240332+0000","last_peered":"2024-04-23T01:49:31.240332+0000","last_clean":"2024-04-23T01:49:31.240332+0000","last_became_active":"2024-04-23T01:48:52.496130+0000","last_became_peered":"2024-04-23T01:48:52.496130+0000","last_unstale":"2024-04-23T01:49:31.240332+0000","last_undegraded":"2024-04-23T01:49:31.240332+0000","last_fullsized":"2024-04-23T01:49:31.240332+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3407872,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":3328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.346665+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:48:53.346665+0000","last_peered":"2024-04-23T01:48:53.346665+0000","last_clean":"2024-04-23T01:48:53.346665+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:48:53.346665+0000","last_undegraded":"2024-04-23T01:48:53.346665+0000","last_fullsized":"2024-04-23T01:48:53.346665+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"36'327","reported_seq":340,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:30.971074+0000","last_change":"2024-04-23T01:48:52.473323+0000","last_active":"2024-04-23T01:49:30.971074+0000","last_peered":"2024-04-23T01:49:30.971074+0000","last_clean":"2024-04-23T01:49:30.971074+0000","last_became_active":"2024-04-23T01:48:52.466602+0000","last_became_peered":"2024-04-23T01:48:52.466602+0000","last_unstale":"2024-04-23T01:49:30.971074+0000","last_undegraded":"2024-04-23T01:49:30.971074+0000","last_fullsized":"2024-04-23T01:49:30.971074+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2686976,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":328,"num_write_kb":2624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.439757+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:48:53.439757+0000","last_peered":"2024-04-23T01:48:53.439757+0000","last_clean":"2024-04-23T01:48:53.439757+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:48:53.439757+0000","last_undegraded":"2024-04-23T01:48:53.439757+0000","last_fullsized":"2024-04-23T01:48:53.439757+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"36'324","reported_seq":337,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:31.578218+0000","last_change":"2024-04-23T01:48:52.464660+0000","last_active":"2024-04-23T01:49:31.578218+0000","last_peered":"2024-04-23T01:49:31.578218+0000","last_clean":"2024-04-23T01:49:31.578218+0000","last_became_active":"2024-04-23T01:48:52.455507+0000","last_became_peered":"2024-04-23T01:48:52.455507+0000","last_unstale":"2024-04-23T01:49:31.578218+0000","last_undegraded":"2024-04-23T01:49:31.578218+0000","last_fullsized":"2024-04-23T01:49:31.578218+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2662400,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":325,"num_write_kb":2600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.430351+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:48:53.430351+0000","last_peered":"2024-04-23T01:48:53.430351+0000","last_clean":"2024-04-23T01:48:53.430351+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:48:53.430351+0000","last_undegraded":"2024-04-23T01:48:53.430351+0000","last_fullsized":"2024-04-23T01:48:53.430351+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":47570944,"num_objects":728,"num_object_clones":0,"num_object_copies":2184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5807,"num_write_kb":46456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":71340032,"data_stored":142680064,"data_compressed":7364061,"data_compressed_allocated":71340032,"data_compressed_original":142680064,"omap_allocated":0,"internal_metadata":0},"log_size":5791,"ondisk_log_size":5791,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705928,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":61264,"kb_used_data":16392,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94310576,"statfs":{"total":96636764160,"available":96574029824,"internally_reserved":0,"allocated":16785408,"data_stored":33528818,"data_compressed":1724517,"data_compressed_allocated":16457728,"data_compressed_original":33406976,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705928,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65104,"kb_used_data":20232,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94306736,"statfs":{"total":96636764160,"available":96570097664,"internally_reserved":0,"allocated":20717568,"data_stored":41393138,"data_compressed":2130467,"data_compressed_allocated":20389888,"data_compressed_original":41271296,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705928,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":67524,"kb_used_data":14332,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94304316,"statfs":{"total":96636764160,"available":96567619584,"internally_reserved":0,"allocated":14675968,"data_stored":28842450,"data_compressed":1483178,"data_compressed_allocated":14368768,"data_compressed_original":28737536,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705928,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":67788,"kb_used_data":18756,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94304052,"statfs":{"total":96636764160,"available":96567349248,"internally_reserved":0,"allocated":19206144,"data_stored":38370290,"data_compressed":1974476,"data_compressed_allocated":18878464,"data_compressed_original":38248448,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":19156992,"data_stored":38313984,"data_compressed":1977517,"data_compressed_allocated":19156992,"data_compressed_original":38313984,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14675968,"data_stored":29351936,"data_compressed":1514903,"data_compressed_allocated":14675968,"data_compressed_original":29351936,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20733952,"data_stored":41467904,"data_compressed":2140276,"data_compressed_allocated":20733952,"data_compressed_original":41467904,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16773120,"data_stored":33546240,"data_compressed":1731365,"data_compressed_allocated":16773120,"data_compressed_original":33546240,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:49:36.406 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 43 16 6519 6503 1.1814 1 0.272073 0.10556 2024-04-23T01:49:37.406 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 44 16 6656 6640 1.17888 1.07031 0.153619 0.105884 2024-04-23T01:49:37.545 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:49:37.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:37.667+0000 7f04cc66d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:37.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:37.673+0000 7f04cc66d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:38.406 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 45 16 6807 6791 1.17889 1.17969 0.128839 0.105876 2024-04-23T01:49:39.406 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 46 16 6956 6940 1.17857 1.16406 0.14793 0.105904 2024-04-23T01:49:39.875 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:49:40.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:40.014+0000 7fab01aed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:40.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:40.021+0000 7fab01aed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:40.406 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 47 16 7102 7086 1.17776 1.14062 0.0963919 0.105983 2024-04-23T01:49:41.358 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:49:41.358 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:49:41.359 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:49:41.406 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 48 16 7249 7233 1.17715 1.14844 0.0329619 0.106031 2024-04-23T01:49:41.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:41.477+0000 7fc612b9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:41.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:41.488+0000 7fc612b9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:41.700 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:49:41.700 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:49:41.716 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":167,"stamp":"2024-04-23T01:49:41.601289+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":54100512,"num_objects":820,"num_object_clones":0,"num_object_copies":2460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":6669,"num_write_kb":54150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6613,"ondisk_log_size":6613,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":287408,"kb_used_data":78800,"kb_used_omap":0,"kb_used_meta":208576,"kb_avail":377199952,"statfs":{"total":386547056640,"available":386252750848,"internally_reserved":0,"allocated":80691200,"data_stored":160746920,"data_compressed":8273694,"data_compressed_allocated":79400960,"data_compressed_original":160276480,"omap_allocated":0,"internal_metadata":213581824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11911168,"num_objects":182,"num_object_clones":0,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1454,"num_write_kb":11632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002253"},"pg_stats":[{"pgid":"5.f","version":"36'375","reported_seq":388,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:35.869926+0000","last_change":"2024-04-23T01:48:52.530638+0000","last_active":"2024-04-23T01:49:35.869926+0000","last_peered":"2024-04-23T01:49:35.869926+0000","last_clean":"2024-04-23T01:49:35.869926+0000","last_became_active":"2024-04-23T01:48:52.520759+0000","last_became_peered":"2024-04-23T01:48:52.520759+0000","last_unstale":"2024-04-23T01:49:35.869926+0000","last_undegraded":"2024-04-23T01:49:35.869926+0000","last_fullsized":"2024-04-23T01:49:35.869926+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":3008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"36'383","reported_seq":396,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:36.491564+0000","last_change":"2024-04-23T01:48:52.532479+0000","last_active":"2024-04-23T01:49:36.491564+0000","last_peered":"2024-04-23T01:49:36.491564+0000","last_clean":"2024-04-23T01:49:36.491564+0000","last_became_active":"2024-04-23T01:48:52.525595+0000","last_became_peered":"2024-04-23T01:48:52.525595+0000","last_unstale":"2024-04-23T01:49:36.491564+0000","last_undegraded":"2024-04-23T01:49:36.491564+0000","last_fullsized":"2024-04-23T01:49:36.491564+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":384,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"36'519","reported_seq":532,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:35.348483+0000","last_change":"2024-04-23T01:48:52.571199+0000","last_active":"2024-04-23T01:49:35.348483+0000","last_peered":"2024-04-23T01:49:35.348483+0000","last_clean":"2024-04-23T01:49:35.348483+0000","last_became_active":"2024-04-23T01:48:52.566277+0000","last_became_peered":"2024-04-23T01:48:52.566277+0000","last_unstale":"2024-04-23T01:49:35.348483+0000","last_undegraded":"2024-04-23T01:49:35.348483+0000","last_fullsized":"2024-04-23T01:49:35.348483+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259840,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":520,"num_write_kb":4160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"36'335","reported_seq":348,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:35.703244+0000","last_change":"2024-04-23T01:48:52.497141+0000","last_active":"2024-04-23T01:49:35.703244+0000","last_peered":"2024-04-23T01:49:35.703244+0000","last_clean":"2024-04-23T01:49:35.703244+0000","last_became_active":"2024-04-23T01:48:52.489329+0000","last_became_peered":"2024-04-23T01:48:52.489329+0000","last_unstale":"2024-04-23T01:49:35.703244+0000","last_undegraded":"2024-04-23T01:49:35.703244+0000","last_fullsized":"2024-04-23T01:49:35.703244+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"36'367","reported_seq":380,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:35.774217+0000","last_change":"2024-04-23T01:48:52.534975+0000","last_active":"2024-04-23T01:49:35.774217+0000","last_peered":"2024-04-23T01:49:35.774217+0000","last_clean":"2024-04-23T01:49:35.774217+0000","last_became_active":"2024-04-23T01:48:52.530664+0000","last_became_peered":"2024-04-23T01:48:52.530664+0000","last_unstale":"2024-04-23T01:49:35.774217+0000","last_undegraded":"2024-04-23T01:49:35.774217+0000","last_fullsized":"2024-04-23T01:49:35.774217+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3014656,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":368,"num_write_kb":2944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"36'407","reported_seq":420,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:35.797344+0000","last_change":"2024-04-23T01:48:52.451606+0000","last_active":"2024-04-23T01:49:35.797344+0000","last_peered":"2024-04-23T01:49:35.797344+0000","last_clean":"2024-04-23T01:49:35.797344+0000","last_became_active":"2024-04-23T01:48:52.439020+0000","last_became_peered":"2024-04-23T01:48:52.439020+0000","last_unstale":"2024-04-23T01:49:35.797344+0000","last_undegraded":"2024-04-23T01:49:35.797344+0000","last_fullsized":"2024-04-23T01:49:35.797344+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342336,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":408,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"36'359","reported_seq":372,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:36.533128+0000","last_change":"2024-04-23T01:48:52.481437+0000","last_active":"2024-04-23T01:49:36.533128+0000","last_peered":"2024-04-23T01:49:36.533128+0000","last_clean":"2024-04-23T01:49:36.533128+0000","last_became_active":"2024-04-23T01:48:52.473748+0000","last_became_peered":"2024-04-23T01:48:52.473748+0000","last_unstale":"2024-04-23T01:49:36.533128+0000","last_undegraded":"2024-04-23T01:49:36.533128+0000","last_fullsized":"2024-04-23T01:49:36.533128+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2949120,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":360,"num_write_kb":2880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"36'407","reported_seq":420,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:36.386662+0000","last_change":"2024-04-23T01:48:52.568521+0000","last_active":"2024-04-23T01:49:36.386662+0000","last_peered":"2024-04-23T01:49:36.386662+0000","last_clean":"2024-04-23T01:49:36.386662+0000","last_became_active":"2024-04-23T01:48:52.562008+0000","last_became_peered":"2024-04-23T01:48:52.562008+0000","last_unstale":"2024-04-23T01:49:36.386662+0000","last_undegraded":"2024-04-23T01:49:36.386662+0000","last_fullsized":"2024-04-23T01:49:36.386662+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342336,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":408,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"36'399","reported_seq":412,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:35.952302+0000","last_change":"2024-04-23T01:48:52.491272+0000","last_active":"2024-04-23T01:49:35.952302+0000","last_peered":"2024-04-23T01:49:35.952302+0000","last_clean":"2024-04-23T01:49:35.952302+0000","last_became_active":"2024-04-23T01:48:52.482878+0000","last_became_peered":"2024-04-23T01:48:52.482878+0000","last_unstale":"2024-04-23T01:49:35.952302+0000","last_undegraded":"2024-04-23T01:49:35.952302+0000","last_fullsized":"2024-04-23T01:49:35.952302+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3276800,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":400,"num_write_kb":3200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":203,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.336526+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:48:53.336526+0000","last_peered":"2024-04-23T01:48:53.336526+0000","last_clean":"2024-04-23T01:48:53.336526+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:48:53.336526+0000","last_undegraded":"2024-04-23T01:48:53.336526+0000","last_fullsized":"2024-04-23T01:48:53.336526+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.441413+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:48:53.441413+0000","last_peered":"2024-04-23T01:48:53.441413+0000","last_clean":"2024-04-23T01:48:53.441413+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:48:53.441413+0000","last_undegraded":"2024-04-23T01:48:53.441413+0000","last_fullsized":"2024-04-23T01:48:53.441413+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"36'439","reported_seq":452,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:35.886424+0000","last_change":"2024-04-23T01:48:52.550172+0000","last_active":"2024-04-23T01:49:35.886424+0000","last_peered":"2024-04-23T01:49:35.886424+0000","last_clean":"2024-04-23T01:49:35.886424+0000","last_became_active":"2024-04-23T01:48:52.541104+0000","last_became_peered":"2024-04-23T01:48:52.541104+0000","last_unstale":"2024-04-23T01:49:35.886424+0000","last_undegraded":"2024-04-23T01:49:35.886424+0000","last_fullsized":"2024-04-23T01:49:35.886424+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3604480,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":440,"num_write_kb":3520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.571998+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:48:53.571998+0000","last_peered":"2024-04-23T01:48:53.571998+0000","last_clean":"2024-04-23T01:48:53.571998+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:48:53.571998+0000","last_undegraded":"2024-04-23T01:48:53.571998+0000","last_fullsized":"2024-04-23T01:48:53.571998+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"36'391","reported_seq":404,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:34.828193+0000","last_change":"2024-04-23T01:48:52.526429+0000","last_active":"2024-04-23T01:49:34.828193+0000","last_peered":"2024-04-23T01:49:34.828193+0000","last_clean":"2024-04-23T01:49:34.828193+0000","last_became_active":"2024-04-23T01:48:52.514040+0000","last_became_peered":"2024-04-23T01:48:52.514040+0000","last_unstale":"2024-04-23T01:49:34.828193+0000","last_undegraded":"2024-04-23T01:49:34.828193+0000","last_fullsized":"2024-04-23T01:49:34.828193+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3211264,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":392,"num_write_kb":3136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.432658+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:48:53.432658+0000","last_peered":"2024-04-23T01:48:53.432658+0000","last_clean":"2024-04-23T01:48:53.432658+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:48:53.432658+0000","last_undegraded":"2024-04-23T01:48:53.432658+0000","last_fullsized":"2024-04-23T01:48:53.432658+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"36'583","reported_seq":596,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:36.144568+0000","last_change":"2024-04-23T01:48:52.469625+0000","last_active":"2024-04-23T01:49:36.144568+0000","last_peered":"2024-04-23T01:49:36.144568+0000","last_clean":"2024-04-23T01:49:36.144568+0000","last_became_active":"2024-04-23T01:48:52.456212+0000","last_became_peered":"2024-04-23T01:48:52.456212+0000","last_unstale":"2024-04-23T01:49:36.144568+0000","last_undegraded":"2024-04-23T01:49:36.144568+0000","last_fullsized":"2024-04-23T01:49:36.144568+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4784128,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":584,"num_write_kb":4672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":68,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.605161+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:48:53.605161+0000","last_peered":"2024-04-23T01:48:53.605161+0000","last_clean":"2024-04-23T01:48:53.605161+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:48:53.605161+0000","last_undegraded":"2024-04-23T01:48:53.605161+0000","last_fullsized":"2024-04-23T01:48:53.605161+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"36'391","reported_seq":404,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:35.642821+0000","last_change":"2024-04-23T01:48:52.505215+0000","last_active":"2024-04-23T01:49:35.642821+0000","last_peered":"2024-04-23T01:49:35.642821+0000","last_clean":"2024-04-23T01:49:35.642821+0000","last_became_active":"2024-04-23T01:48:52.497519+0000","last_became_peered":"2024-04-23T01:48:52.497519+0000","last_unstale":"2024-04-23T01:49:35.642821+0000","last_undegraded":"2024-04-23T01:49:35.642821+0000","last_fullsized":"2024-04-23T01:49:35.642821+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3211264,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":392,"num_write_kb":3136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.438431+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:48:53.438431+0000","last_peered":"2024-04-23T01:48:53.438431+0000","last_clean":"2024-04-23T01:48:53.438431+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:48:53.438431+0000","last_undegraded":"2024-04-23T01:48:53.438431+0000","last_fullsized":"2024-04-23T01:48:53.438431+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"36'439","reported_seq":452,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:34.464329+0000","last_change":"2024-04-23T01:48:52.501817+0000","last_active":"2024-04-23T01:49:34.464329+0000","last_peered":"2024-04-23T01:49:34.464329+0000","last_clean":"2024-04-23T01:49:34.464329+0000","last_became_active":"2024-04-23T01:48:52.496130+0000","last_became_peered":"2024-04-23T01:48:52.496130+0000","last_unstale":"2024-04-23T01:49:34.464329+0000","last_undegraded":"2024-04-23T01:49:34.464329+0000","last_fullsized":"2024-04-23T01:49:34.464329+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3604480,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":440,"num_write_kb":3520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.346665+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:48:53.346665+0000","last_peered":"2024-04-23T01:48:53.346665+0000","last_clean":"2024-04-23T01:48:53.346665+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:48:53.346665+0000","last_undegraded":"2024-04-23T01:48:53.346665+0000","last_fullsized":"2024-04-23T01:48:53.346665+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"36'346","reported_seq":359,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:36.577811+0000","last_change":"2024-04-23T01:48:52.473323+0000","last_active":"2024-04-23T01:49:36.577811+0000","last_peered":"2024-04-23T01:49:36.577811+0000","last_clean":"2024-04-23T01:49:36.577811+0000","last_became_active":"2024-04-23T01:48:52.466602+0000","last_became_peered":"2024-04-23T01:48:52.466602+0000","last_unstale":"2024-04-23T01:49:36.577811+0000","last_undegraded":"2024-04-23T01:49:36.577811+0000","last_fullsized":"2024-04-23T01:49:36.577811+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2842624,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":347,"num_write_kb":2776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.439757+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:48:53.439757+0000","last_peered":"2024-04-23T01:48:53.439757+0000","last_clean":"2024-04-23T01:48:53.439757+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:48:53.439757+0000","last_undegraded":"2024-04-23T01:48:53.439757+0000","last_fullsized":"2024-04-23T01:48:53.439757+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"36'376","reported_seq":389,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:36.592885+0000","last_change":"2024-04-23T01:48:52.464660+0000","last_active":"2024-04-23T01:49:36.592885+0000","last_peered":"2024-04-23T01:49:36.592885+0000","last_clean":"2024-04-23T01:49:36.592885+0000","last_became_active":"2024-04-23T01:48:52.455507+0000","last_became_peered":"2024-04-23T01:48:52.455507+0000","last_unstale":"2024-04-23T01:49:36.592885+0000","last_undegraded":"2024-04-23T01:49:36.592885+0000","last_fullsized":"2024-04-23T01:49:36.592885+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3088384,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":377,"num_write_kb":3016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.430351+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:48:53.430351+0000","last_peered":"2024-04-23T01:48:53.430351+0000","last_clean":"2024-04-23T01:48:53.430351+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:48:53.430351+0000","last_undegraded":"2024-04-23T01:48:53.430351+0000","last_fullsized":"2024-04-23T01:48:53.430351+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":53510144,"num_objects":818,"num_object_clones":0,"num_object_copies":2454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6532,"num_write_kb":52256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":80261120,"data_stored":160522240,"data_compressed":8285355,"data_compressed_allocated":80261120,"data_compressed_original":160522240,"omap_allocated":0,"internal_metadata":0},"log_size":6516,"ondisk_log_size":6516,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":67464,"kb_used_data":18432,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94304376,"statfs":{"total":96636764160,"available":96567681024,"internally_reserved":0,"allocated":18874368,"data_stored":37706738,"data_compressed":1940247,"data_compressed_allocated":18546688,"data_compressed_original":37584896,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":71944,"kb_used_data":22912,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94299896,"statfs":{"total":96636764160,"available":96563093504,"internally_reserved":0,"allocated":23461888,"data_stored":46881778,"data_compressed":2413877,"data_compressed_allocated":23134208,"data_compressed_original":46759936,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":73800,"kb_used_data":16448,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94298040,"statfs":{"total":96636764160,"available":96561192960,"internally_reserved":0,"allocated":16842752,"data_stored":33176018,"data_compressed":1706945,"data_compressed_allocated":16535552,"data_compressed_original":33071104,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":74200,"kb_used_data":21008,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94297640,"statfs":{"total":96636764160,"available":96560783360,"internally_reserved":0,"allocated":21512192,"data_stored":42982386,"data_compressed":2212625,"data_compressed_allocated":21184512,"data_compressed_original":42860544,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":21381120,"data_stored":42762240,"data_compressed":2207206,"data_compressed_allocated":21381120,"data_compressed_original":42762240,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16818176,"data_stored":33636352,"data_compressed":1736132,"data_compressed_allocated":16818176,"data_compressed_original":33636352,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":23437312,"data_stored":46874624,"data_compressed":2419456,"data_compressed_allocated":23437312,"data_compressed_original":46874624,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":18624512,"data_stored":37249024,"data_compressed":1922561,"data_compressed_allocated":18624512,"data_compressed_original":37249024,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:49:42.223 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:49:42.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:42.338+0000 7f6f7c15d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:42.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:42.344+0000 7f6f7c15d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:42.406 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 49 16 7371 7355 1.17257 0.953125 0.0897595 0.106431 2024-04-23T01:49:43.407 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 50 16 7519 7503 1.17224 1.15625 0.0942503 0.106483 2024-04-23T01:49:44.407 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 51 16 7668 7652 1.17208 1.16406 0.153477 0.10651 2024-04-23T01:49:44.630 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:49:44.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:44.741+0000 7fbfbac6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:44.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:44.749+0000 7fbfbac6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:45.407 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 52 16 7803 7787 1.16982 1.05469 0.0665799 0.106705 2024-04-23T01:49:46.407 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 53 16 7946 7930 1.16883 1.11719 0.0346229 0.106818 2024-04-23T01:49:46.717 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:49:46.718 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:49:46.718 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:49:46.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:46.839+0000 7f984215d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:46.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:46.848+0000 7f984215d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:46.895 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:49:47.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:47.019+0000 7f8600e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:47.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:47.029+0000 7f8600e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:47.175 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:49:47.175 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:49:47.190 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-04-23T01:49:45.602106+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":60047904,"num_objects":910,"num_object_clones":0,"num_object_copies":2730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":7395,"num_write_kb":59958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7339,"ondisk_log_size":7339,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":295908,"kb_used_data":87300,"kb_used_omap":0,"kb_used_meta":208576,"kb_avail":377191452,"statfs":{"total":386547056640,"available":386244046848,"internally_reserved":0,"allocated":89395200,"data_stored":178154920,"data_compressed":9172569,"data_compressed_allocated":88104960,"data_compressed_original":177684480,"omap_allocated":0,"internal_metadata":213581824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11886592,"num_objects":180,"num_object_clones":0,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1451,"num_write_kb":11608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002408"},"pg_stats":[{"pgid":"5.f","version":"36'439","reported_seq":452,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:41.459002+0000","last_change":"2024-04-23T01:48:52.530638+0000","last_active":"2024-04-23T01:49:41.459002+0000","last_peered":"2024-04-23T01:49:41.459002+0000","last_clean":"2024-04-23T01:49:41.459002+0000","last_became_active":"2024-04-23T01:48:52.520759+0000","last_became_peered":"2024-04-23T01:48:52.520759+0000","last_unstale":"2024-04-23T01:49:41.459002+0000","last_undegraded":"2024-04-23T01:49:41.459002+0000","last_fullsized":"2024-04-23T01:49:41.459002+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3604480,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":440,"num_write_kb":3520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"36'447","reported_seq":460,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:40.114682+0000","last_change":"2024-04-23T01:48:52.532479+0000","last_active":"2024-04-23T01:49:40.114682+0000","last_peered":"2024-04-23T01:49:40.114682+0000","last_clean":"2024-04-23T01:49:40.114682+0000","last_became_active":"2024-04-23T01:48:52.525595+0000","last_became_peered":"2024-04-23T01:48:52.525595+0000","last_unstale":"2024-04-23T01:49:40.114682+0000","last_undegraded":"2024-04-23T01:49:40.114682+0000","last_fullsized":"2024-04-23T01:49:40.114682+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":448,"num_write_kb":3584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"36'575","reported_seq":588,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:41.332042+0000","last_change":"2024-04-23T01:48:52.571199+0000","last_active":"2024-04-23T01:49:41.332042+0000","last_peered":"2024-04-23T01:49:41.332042+0000","last_clean":"2024-04-23T01:49:41.332042+0000","last_became_active":"2024-04-23T01:48:52.566277+0000","last_became_peered":"2024-04-23T01:48:52.566277+0000","last_unstale":"2024-04-23T01:49:41.332042+0000","last_undegraded":"2024-04-23T01:49:41.332042+0000","last_fullsized":"2024-04-23T01:49:41.332042+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718592,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":576,"num_write_kb":4608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"36'391","reported_seq":404,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:41.145237+0000","last_change":"2024-04-23T01:48:52.497141+0000","last_active":"2024-04-23T01:49:41.145237+0000","last_peered":"2024-04-23T01:49:41.145237+0000","last_clean":"2024-04-23T01:49:41.145237+0000","last_became_active":"2024-04-23T01:48:52.489329+0000","last_became_peered":"2024-04-23T01:48:52.489329+0000","last_unstale":"2024-04-23T01:49:41.145237+0000","last_undegraded":"2024-04-23T01:49:41.145237+0000","last_fullsized":"2024-04-23T01:49:41.145237+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3211264,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":392,"num_write_kb":3136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"36'391","reported_seq":404,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:39.234675+0000","last_change":"2024-04-23T01:48:52.534975+0000","last_active":"2024-04-23T01:49:39.234675+0000","last_peered":"2024-04-23T01:49:39.234675+0000","last_clean":"2024-04-23T01:49:39.234675+0000","last_became_active":"2024-04-23T01:48:52.530664+0000","last_became_peered":"2024-04-23T01:48:52.530664+0000","last_unstale":"2024-04-23T01:49:39.234675+0000","last_undegraded":"2024-04-23T01:49:39.234675+0000","last_fullsized":"2024-04-23T01:49:39.234675+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3211264,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":392,"num_write_kb":3136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"36'447","reported_seq":460,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:38.585517+0000","last_change":"2024-04-23T01:48:52.451606+0000","last_active":"2024-04-23T01:49:38.585517+0000","last_peered":"2024-04-23T01:49:38.585517+0000","last_clean":"2024-04-23T01:49:38.585517+0000","last_became_active":"2024-04-23T01:48:52.439020+0000","last_became_peered":"2024-04-23T01:48:52.439020+0000","last_unstale":"2024-04-23T01:49:38.585517+0000","last_undegraded":"2024-04-23T01:49:38.585517+0000","last_fullsized":"2024-04-23T01:49:38.585517+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":448,"num_write_kb":3584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"36'399","reported_seq":412,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:41.076955+0000","last_change":"2024-04-23T01:48:52.481437+0000","last_active":"2024-04-23T01:49:41.076955+0000","last_peered":"2024-04-23T01:49:41.076955+0000","last_clean":"2024-04-23T01:49:41.076955+0000","last_became_active":"2024-04-23T01:48:52.473748+0000","last_became_peered":"2024-04-23T01:48:52.473748+0000","last_unstale":"2024-04-23T01:49:41.076955+0000","last_undegraded":"2024-04-23T01:49:41.076955+0000","last_fullsized":"2024-04-23T01:49:41.076955+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3276800,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":400,"num_write_kb":3200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"36'454","reported_seq":467,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:41.546165+0000","last_change":"2024-04-23T01:48:52.568521+0000","last_active":"2024-04-23T01:49:41.546165+0000","last_peered":"2024-04-23T01:49:41.546165+0000","last_clean":"2024-04-23T01:49:41.546165+0000","last_became_active":"2024-04-23T01:48:52.562008+0000","last_became_peered":"2024-04-23T01:48:52.562008+0000","last_unstale":"2024-04-23T01:49:41.546165+0000","last_undegraded":"2024-04-23T01:49:41.546165+0000","last_fullsized":"2024-04-23T01:49:41.546165+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3727360,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":455,"num_write_kb":3640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"36'463","reported_seq":476,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:40.156911+0000","last_change":"2024-04-23T01:48:52.491272+0000","last_active":"2024-04-23T01:49:40.156911+0000","last_peered":"2024-04-23T01:49:40.156911+0000","last_clean":"2024-04-23T01:49:40.156911+0000","last_became_active":"2024-04-23T01:48:52.482878+0000","last_became_peered":"2024-04-23T01:48:52.482878+0000","last_unstale":"2024-04-23T01:49:40.156911+0000","last_undegraded":"2024-04-23T01:49:40.156911+0000","last_fullsized":"2024-04-23T01:49:40.156911+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":464,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":203,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.336526+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:48:53.336526+0000","last_peered":"2024-04-23T01:48:53.336526+0000","last_clean":"2024-04-23T01:48:53.336526+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:48:53.336526+0000","last_undegraded":"2024-04-23T01:48:53.336526+0000","last_fullsized":"2024-04-23T01:48:53.336526+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.441413+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:48:53.441413+0000","last_peered":"2024-04-23T01:48:53.441413+0000","last_clean":"2024-04-23T01:48:53.441413+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:48:53.441413+0000","last_undegraded":"2024-04-23T01:48:53.441413+0000","last_fullsized":"2024-04-23T01:48:53.441413+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"36'479","reported_seq":492,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:41.172498+0000","last_change":"2024-04-23T01:48:52.550172+0000","last_active":"2024-04-23T01:49:41.172498+0000","last_peered":"2024-04-23T01:49:41.172498+0000","last_clean":"2024-04-23T01:49:41.172498+0000","last_became_active":"2024-04-23T01:48:52.541104+0000","last_became_peered":"2024-04-23T01:48:52.541104+0000","last_unstale":"2024-04-23T01:49:41.172498+0000","last_undegraded":"2024-04-23T01:49:41.172498+0000","last_fullsized":"2024-04-23T01:49:41.172498+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":480,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.571998+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:48:53.571998+0000","last_peered":"2024-04-23T01:48:53.571998+0000","last_clean":"2024-04-23T01:48:53.571998+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:48:53.571998+0000","last_undegraded":"2024-04-23T01:48:53.571998+0000","last_fullsized":"2024-04-23T01:48:53.571998+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"36'423","reported_seq":436,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:37.695917+0000","last_change":"2024-04-23T01:48:52.526429+0000","last_active":"2024-04-23T01:49:37.695917+0000","last_peered":"2024-04-23T01:49:37.695917+0000","last_clean":"2024-04-23T01:49:37.695917+0000","last_became_active":"2024-04-23T01:48:52.514040+0000","last_became_peered":"2024-04-23T01:48:52.514040+0000","last_unstale":"2024-04-23T01:49:37.695917+0000","last_undegraded":"2024-04-23T01:49:37.695917+0000","last_fullsized":"2024-04-23T01:49:37.695917+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3473408,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":424,"num_write_kb":3392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.432658+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:48:53.432658+0000","last_peered":"2024-04-23T01:48:53.432658+0000","last_clean":"2024-04-23T01:48:53.432658+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:48:53.432658+0000","last_undegraded":"2024-04-23T01:48:53.432658+0000","last_fullsized":"2024-04-23T01:48:53.432658+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"36'647","reported_seq":660,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:41.114677+0000","last_change":"2024-04-23T01:48:52.469625+0000","last_active":"2024-04-23T01:49:41.114677+0000","last_peered":"2024-04-23T01:49:41.114677+0000","last_clean":"2024-04-23T01:49:41.114677+0000","last_became_active":"2024-04-23T01:48:52.456212+0000","last_became_peered":"2024-04-23T01:48:52.456212+0000","last_unstale":"2024-04-23T01:49:41.114677+0000","last_undegraded":"2024-04-23T01:49:41.114677+0000","last_fullsized":"2024-04-23T01:49:41.114677+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308416,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":648,"num_write_kb":5184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":68,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.605161+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:48:53.605161+0000","last_peered":"2024-04-23T01:48:53.605161+0000","last_clean":"2024-04-23T01:48:53.605161+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:48:53.605161+0000","last_undegraded":"2024-04-23T01:48:53.605161+0000","last_fullsized":"2024-04-23T01:48:53.605161+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"36'439","reported_seq":452,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:41.270638+0000","last_change":"2024-04-23T01:48:52.505215+0000","last_active":"2024-04-23T01:49:41.270638+0000","last_peered":"2024-04-23T01:49:41.270638+0000","last_clean":"2024-04-23T01:49:41.270638+0000","last_became_active":"2024-04-23T01:48:52.497519+0000","last_became_peered":"2024-04-23T01:48:52.497519+0000","last_unstale":"2024-04-23T01:49:41.270638+0000","last_undegraded":"2024-04-23T01:49:41.270638+0000","last_fullsized":"2024-04-23T01:49:41.270638+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3604480,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":440,"num_write_kb":3520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.438431+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:48:53.438431+0000","last_peered":"2024-04-23T01:48:53.438431+0000","last_clean":"2024-04-23T01:48:53.438431+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:48:53.438431+0000","last_undegraded":"2024-04-23T01:48:53.438431+0000","last_fullsized":"2024-04-23T01:48:53.438431+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"36'471","reported_seq":484,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:40.321754+0000","last_change":"2024-04-23T01:48:52.501817+0000","last_active":"2024-04-23T01:49:40.321754+0000","last_peered":"2024-04-23T01:49:40.321754+0000","last_clean":"2024-04-23T01:49:40.321754+0000","last_became_active":"2024-04-23T01:48:52.496130+0000","last_became_peered":"2024-04-23T01:48:52.496130+0000","last_unstale":"2024-04-23T01:49:40.321754+0000","last_undegraded":"2024-04-23T01:49:40.321754+0000","last_fullsized":"2024-04-23T01:49:40.321754+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3866624,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":472,"num_write_kb":3776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.346665+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:48:53.346665+0000","last_peered":"2024-04-23T01:48:53.346665+0000","last_clean":"2024-04-23T01:48:53.346665+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:48:53.346665+0000","last_undegraded":"2024-04-23T01:48:53.346665+0000","last_fullsized":"2024-04-23T01:48:53.346665+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"36'374","reported_seq":387,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:41.515367+0000","last_change":"2024-04-23T01:48:52.473323+0000","last_active":"2024-04-23T01:49:41.515367+0000","last_peered":"2024-04-23T01:49:41.515367+0000","last_clean":"2024-04-23T01:49:41.515367+0000","last_became_active":"2024-04-23T01:48:52.466602+0000","last_became_peered":"2024-04-23T01:48:52.466602+0000","last_unstale":"2024-04-23T01:49:41.515367+0000","last_undegraded":"2024-04-23T01:49:41.515367+0000","last_fullsized":"2024-04-23T01:49:41.515367+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3072000,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":375,"num_write_kb":3000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.439757+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:48:53.439757+0000","last_peered":"2024-04-23T01:48:53.439757+0000","last_clean":"2024-04-23T01:48:53.439757+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:48:53.439757+0000","last_undegraded":"2024-04-23T01:48:53.439757+0000","last_fullsized":"2024-04-23T01:48:53.439757+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"36'403","reported_seq":416,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:41.559612+0000","last_change":"2024-04-23T01:48:52.464660+0000","last_active":"2024-04-23T01:49:41.559612+0000","last_peered":"2024-04-23T01:49:41.559612+0000","last_clean":"2024-04-23T01:49:41.559612+0000","last_became_active":"2024-04-23T01:48:52.455507+0000","last_became_peered":"2024-04-23T01:48:52.455507+0000","last_unstale":"2024-04-23T01:49:41.559612+0000","last_undegraded":"2024-04-23T01:49:41.559612+0000","last_fullsized":"2024-04-23T01:49:41.559612+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3309568,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":404,"num_write_kb":3232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.430351+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:48:53.430351+0000","last_peered":"2024-04-23T01:48:53.430351+0000","last_clean":"2024-04-23T01:48:53.430351+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:48:53.430351+0000","last_undegraded":"2024-04-23T01:48:53.430351+0000","last_fullsized":"2024-04-23T01:48:53.430351+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":59457536,"num_objects":908,"num_object_clones":0,"num_object_copies":2724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7258,"num_write_kb":58064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":89186304,"data_stored":178372608,"data_compressed":9207072,"data_compressed_allocated":89186304,"data_compressed_original":178372608,"omap_allocated":0,"internal_metadata":0},"log_size":7242,"ondisk_log_size":7242,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705938,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":69396,"kb_used_data":20364,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94302444,"statfs":{"total":96636764160,"available":96565702656,"internally_reserved":0,"allocated":20852736,"data_stored":41663474,"data_compressed":2144556,"data_compressed_allocated":20525056,"data_compressed_original":41541632,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705938,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":74556,"kb_used_data":25524,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94297284,"statfs":{"total":96636764160,"available":96560418816,"internally_reserved":0,"allocated":26136576,"data_stored":52231154,"data_compressed":2690096,"data_compressed_allocated":25808896,"data_compressed_original":52109312,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705938,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":75552,"kb_used_data":18200,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94296288,"statfs":{"total":96636764160,"available":96559398912,"internally_reserved":0,"allocated":18636800,"data_stored":36764114,"data_compressed":1892219,"data_compressed_allocated":18329600,"data_compressed_original":36659200,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705938,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":76404,"kb_used_data":23212,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94295436,"statfs":{"total":96636764160,"available":96558526464,"internally_reserved":0,"allocated":23769088,"data_stored":47496178,"data_compressed":2445698,"data_compressed_allocated":23441408,"data_compressed_original":47374336,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":23666688,"data_stored":47333376,"data_compressed":2443240,"data_compressed_allocated":23666688,"data_compressed_original":47333376,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":18628608,"data_stored":37257216,"data_compressed":1923098,"data_compressed_allocated":18628608,"data_compressed_original":37257216,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":26185728,"data_stored":52371456,"data_compressed":2703289,"data_compressed_allocated":26185728,"data_compressed_original":52371456,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20705280,"data_stored":41410560,"data_compressed":2137445,"data_compressed_allocated":20705280,"data_compressed_original":41410560,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:49:47.407 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 54 16 8093 8077 1.16845 1.14844 0.102491 0.106882 2024-04-23T01:49:48.407 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 55 16 8224 8208 1.16581 1.02344 0.116976 0.107106 2024-04-23T01:49:49.240 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:49:49.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:49.363+0000 7feba782d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:49.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:49.370+0000 7feba782d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:49.407 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 56 16 8354 8338 1.16313 1.01562 0.0918944 0.107318 2024-04-23T01:49:50.407 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 57 16 8485 8469 1.16068 1.02344 0.0261094 0.107535 2024-04-23T01:49:51.407 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 58 16 8624 8608 1.15938 1.08594 0.148187 0.107667 2024-04-23T01:49:51.580 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:49:51.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:51.701+0000 7ff3c146d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:51.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:51.706+0000 7ff3c146d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:52.192 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:49:52.192 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:49:52.193 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:49:52.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:52.311+0000 7f8333f5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:52.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:52.321+0000 7f8333f5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:52.408 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 59 16 8762 8746 1.15801 1.07812 0.0792547 0.107838 2024-04-23T01:49:52.627 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:49:52.628 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:49:52.644 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2024-04-23T01:49:51.603244+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":65839648,"num_objects":999,"num_object_clones":0,"num_object_copies":2997,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":8102,"num_write_kb":65614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8046,"ondisk_log_size":8046,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":339116,"kb_used_data":95820,"kb_used_omap":0,"kb_used_meta":243264,"kb_avail":377148244,"statfs":{"total":386547056640,"available":386199801856,"internally_reserved":0,"allocated":98119680,"data_stored":195603880,"data_compressed":10073559,"data_compressed_allocated":96829440,"data_compressed_original":195133440,"omap_allocated":0,"internal_metadata":249102336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11739136,"num_objects":179,"num_object_clones":0,"num_object_copies":537,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1433,"num_write_kb":11464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002309"},"pg_stats":[{"pgid":"5.f","version":"36'495","reported_seq":508,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:46.463369+0000","last_change":"2024-04-23T01:48:52.530638+0000","last_active":"2024-04-23T01:49:46.463369+0000","last_peered":"2024-04-23T01:49:46.463369+0000","last_clean":"2024-04-23T01:49:46.463369+0000","last_became_active":"2024-04-23T01:48:52.520759+0000","last_became_peered":"2024-04-23T01:48:52.520759+0000","last_unstale":"2024-04-23T01:49:46.463369+0000","last_undegraded":"2024-04-23T01:49:46.463369+0000","last_fullsized":"2024-04-23T01:49:46.463369+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4063232,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":496,"num_write_kb":3968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"36'535","reported_seq":548,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:46.088014+0000","last_change":"2024-04-23T01:48:52.532479+0000","last_active":"2024-04-23T01:49:46.088014+0000","last_peered":"2024-04-23T01:49:46.088014+0000","last_clean":"2024-04-23T01:49:46.088014+0000","last_became_active":"2024-04-23T01:48:52.525595+0000","last_became_peered":"2024-04-23T01:48:52.525595+0000","last_unstale":"2024-04-23T01:49:46.088014+0000","last_undegraded":"2024-04-23T01:49:46.088014+0000","last_fullsized":"2024-04-23T01:49:46.088014+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4390912,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":536,"num_write_kb":4288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"36'615","reported_seq":628,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:45.742490+0000","last_change":"2024-04-23T01:48:52.571199+0000","last_active":"2024-04-23T01:49:45.742490+0000","last_peered":"2024-04-23T01:49:45.742490+0000","last_clean":"2024-04-23T01:49:45.742490+0000","last_became_active":"2024-04-23T01:48:52.566277+0000","last_became_peered":"2024-04-23T01:48:52.566277+0000","last_unstale":"2024-04-23T01:49:45.742490+0000","last_undegraded":"2024-04-23T01:49:45.742490+0000","last_fullsized":"2024-04-23T01:49:45.742490+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5046272,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":616,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"36'450","reported_seq":463,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:46.587289+0000","last_change":"2024-04-23T01:48:52.497141+0000","last_active":"2024-04-23T01:49:46.587289+0000","last_peered":"2024-04-23T01:49:46.587289+0000","last_clean":"2024-04-23T01:49:46.587289+0000","last_became_active":"2024-04-23T01:48:52.489329+0000","last_became_peered":"2024-04-23T01:48:52.489329+0000","last_unstale":"2024-04-23T01:49:46.587289+0000","last_undegraded":"2024-04-23T01:49:46.587289+0000","last_fullsized":"2024-04-23T01:49:46.587289+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3694592,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":451,"num_write_kb":3608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"36'447","reported_seq":460,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:45.764215+0000","last_change":"2024-04-23T01:48:52.534975+0000","last_active":"2024-04-23T01:49:45.764215+0000","last_peered":"2024-04-23T01:49:45.764215+0000","last_clean":"2024-04-23T01:49:45.764215+0000","last_became_active":"2024-04-23T01:48:52.530664+0000","last_became_peered":"2024-04-23T01:48:52.530664+0000","last_unstale":"2024-04-23T01:49:45.764215+0000","last_undegraded":"2024-04-23T01:49:45.764215+0000","last_fullsized":"2024-04-23T01:49:45.764215+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":448,"num_write_kb":3584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"36'487","reported_seq":500,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:46.323915+0000","last_change":"2024-04-23T01:48:52.451606+0000","last_active":"2024-04-23T01:49:46.323915+0000","last_peered":"2024-04-23T01:49:46.323915+0000","last_clean":"2024-04-23T01:49:46.323915+0000","last_became_active":"2024-04-23T01:48:52.439020+0000","last_became_peered":"2024-04-23T01:48:52.439020+0000","last_unstale":"2024-04-23T01:49:46.323915+0000","last_undegraded":"2024-04-23T01:49:46.323915+0000","last_fullsized":"2024-04-23T01:49:46.323915+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3997696,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":488,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"36'439","reported_seq":452,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:46.537438+0000","last_change":"2024-04-23T01:48:52.481437+0000","last_active":"2024-04-23T01:49:46.537438+0000","last_peered":"2024-04-23T01:49:46.537438+0000","last_clean":"2024-04-23T01:49:46.537438+0000","last_became_active":"2024-04-23T01:48:52.473748+0000","last_became_peered":"2024-04-23T01:48:52.473748+0000","last_unstale":"2024-04-23T01:49:46.537438+0000","last_undegraded":"2024-04-23T01:49:46.537438+0000","last_fullsized":"2024-04-23T01:49:46.537438+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3604480,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":440,"num_write_kb":3520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"36'500","reported_seq":513,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:46.592683+0000","last_change":"2024-04-23T01:48:52.568521+0000","last_active":"2024-04-23T01:49:46.592683+0000","last_peered":"2024-04-23T01:49:46.592683+0000","last_clean":"2024-04-23T01:49:46.592683+0000","last_became_active":"2024-04-23T01:48:52.562008+0000","last_became_peered":"2024-04-23T01:48:52.562008+0000","last_unstale":"2024-04-23T01:49:46.592683+0000","last_undegraded":"2024-04-23T01:49:46.592683+0000","last_fullsized":"2024-04-23T01:49:46.592683+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4104192,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":501,"num_write_kb":4008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"36'495","reported_seq":508,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:44.547572+0000","last_change":"2024-04-23T01:48:52.491272+0000","last_active":"2024-04-23T01:49:44.547572+0000","last_peered":"2024-04-23T01:49:44.547572+0000","last_clean":"2024-04-23T01:49:44.547572+0000","last_became_active":"2024-04-23T01:48:52.482878+0000","last_became_peered":"2024-04-23T01:48:52.482878+0000","last_unstale":"2024-04-23T01:49:44.547572+0000","last_undegraded":"2024-04-23T01:49:44.547572+0000","last_fullsized":"2024-04-23T01:49:44.547572+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4063232,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":496,"num_write_kb":3968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":203,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.336526+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:48:53.336526+0000","last_peered":"2024-04-23T01:48:53.336526+0000","last_clean":"2024-04-23T01:48:53.336526+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:48:53.336526+0000","last_undegraded":"2024-04-23T01:48:53.336526+0000","last_fullsized":"2024-04-23T01:48:53.336526+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.441413+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:48:53.441413+0000","last_peered":"2024-04-23T01:48:53.441413+0000","last_clean":"2024-04-23T01:48:53.441413+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:48:53.441413+0000","last_undegraded":"2024-04-23T01:48:53.441413+0000","last_fullsized":"2024-04-23T01:48:53.441413+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"36'519","reported_seq":532,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:46.012411+0000","last_change":"2024-04-23T01:48:52.550172+0000","last_active":"2024-04-23T01:49:46.012411+0000","last_peered":"2024-04-23T01:49:46.012411+0000","last_clean":"2024-04-23T01:49:46.012411+0000","last_became_active":"2024-04-23T01:48:52.541104+0000","last_became_peered":"2024-04-23T01:48:52.541104+0000","last_unstale":"2024-04-23T01:49:46.012411+0000","last_undegraded":"2024-04-23T01:49:46.012411+0000","last_fullsized":"2024-04-23T01:49:46.012411+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259840,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":520,"num_write_kb":4160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.571998+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:48:53.571998+0000","last_peered":"2024-04-23T01:48:53.571998+0000","last_clean":"2024-04-23T01:48:53.571998+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:48:53.571998+0000","last_undegraded":"2024-04-23T01:48:53.571998+0000","last_fullsized":"2024-04-23T01:48:53.571998+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"36'452","reported_seq":465,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:46.565713+0000","last_change":"2024-04-23T01:48:52.526429+0000","last_active":"2024-04-23T01:49:46.565713+0000","last_peered":"2024-04-23T01:49:46.565713+0000","last_clean":"2024-04-23T01:49:46.565713+0000","last_became_active":"2024-04-23T01:48:52.514040+0000","last_became_peered":"2024-04-23T01:48:52.514040+0000","last_unstale":"2024-04-23T01:49:46.565713+0000","last_undegraded":"2024-04-23T01:49:46.565713+0000","last_fullsized":"2024-04-23T01:49:46.565713+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3710976,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":453,"num_write_kb":3624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.432658+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:48:53.432658+0000","last_peered":"2024-04-23T01:48:53.432658+0000","last_clean":"2024-04-23T01:48:53.432658+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:48:53.432658+0000","last_undegraded":"2024-04-23T01:48:53.432658+0000","last_fullsized":"2024-04-23T01:48:53.432658+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"36'671","reported_seq":684,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:45.441647+0000","last_change":"2024-04-23T01:48:52.469625+0000","last_active":"2024-04-23T01:49:45.441647+0000","last_peered":"2024-04-23T01:49:45.441647+0000","last_clean":"2024-04-23T01:49:45.441647+0000","last_became_active":"2024-04-23T01:48:52.456212+0000","last_became_peered":"2024-04-23T01:48:52.456212+0000","last_unstale":"2024-04-23T01:49:45.441647+0000","last_undegraded":"2024-04-23T01:49:45.441647+0000","last_fullsized":"2024-04-23T01:49:45.441647+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505024,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":672,"num_write_kb":5376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":68,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.605161+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:48:53.605161+0000","last_peered":"2024-04-23T01:48:53.605161+0000","last_clean":"2024-04-23T01:48:53.605161+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:48:53.605161+0000","last_undegraded":"2024-04-23T01:48:53.605161+0000","last_fullsized":"2024-04-23T01:48:53.605161+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"36'455","reported_seq":468,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:45.863381+0000","last_change":"2024-04-23T01:48:52.505215+0000","last_active":"2024-04-23T01:49:45.863381+0000","last_peered":"2024-04-23T01:49:45.863381+0000","last_clean":"2024-04-23T01:49:45.863381+0000","last_became_active":"2024-04-23T01:48:52.497519+0000","last_became_peered":"2024-04-23T01:48:52.497519+0000","last_unstale":"2024-04-23T01:49:45.863381+0000","last_undegraded":"2024-04-23T01:49:45.863381+0000","last_fullsized":"2024-04-23T01:49:45.863381+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":456,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.438431+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:48:53.438431+0000","last_peered":"2024-04-23T01:48:53.438431+0000","last_clean":"2024-04-23T01:48:53.438431+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:48:53.438431+0000","last_undegraded":"2024-04-23T01:48:53.438431+0000","last_fullsized":"2024-04-23T01:48:53.438431+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"36'511","reported_seq":524,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:46.250400+0000","last_change":"2024-04-23T01:48:52.501817+0000","last_active":"2024-04-23T01:49:46.250400+0000","last_peered":"2024-04-23T01:49:46.250400+0000","last_clean":"2024-04-23T01:49:46.250400+0000","last_became_active":"2024-04-23T01:48:52.496130+0000","last_became_peered":"2024-04-23T01:48:52.496130+0000","last_unstale":"2024-04-23T01:49:46.250400+0000","last_undegraded":"2024-04-23T01:49:46.250400+0000","last_fullsized":"2024-04-23T01:49:46.250400+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":512,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.346665+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:48:53.346665+0000","last_peered":"2024-04-23T01:48:53.346665+0000","last_clean":"2024-04-23T01:48:53.346665+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:48:53.346665+0000","last_undegraded":"2024-04-23T01:48:53.346665+0000","last_fullsized":"2024-04-23T01:48:53.346665+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"36'423","reported_seq":436,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:45.957610+0000","last_change":"2024-04-23T01:48:52.473323+0000","last_active":"2024-04-23T01:49:45.957610+0000","last_peered":"2024-04-23T01:49:45.957610+0000","last_clean":"2024-04-23T01:49:45.957610+0000","last_became_active":"2024-04-23T01:48:52.466602+0000","last_became_peered":"2024-04-23T01:48:52.466602+0000","last_unstale":"2024-04-23T01:49:45.957610+0000","last_undegraded":"2024-04-23T01:49:45.957610+0000","last_fullsized":"2024-04-23T01:49:45.957610+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3473408,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":424,"num_write_kb":3392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.439757+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:48:53.439757+0000","last_peered":"2024-04-23T01:48:53.439757+0000","last_clean":"2024-04-23T01:48:53.439757+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:48:53.439757+0000","last_undegraded":"2024-04-23T01:48:53.439757+0000","last_fullsized":"2024-04-23T01:48:53.439757+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"36'455","reported_seq":468,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:46.344658+0000","last_change":"2024-04-23T01:48:52.464660+0000","last_active":"2024-04-23T01:49:46.344658+0000","last_peered":"2024-04-23T01:49:46.344658+0000","last_clean":"2024-04-23T01:49:46.344658+0000","last_became_active":"2024-04-23T01:48:52.455507+0000","last_became_peered":"2024-04-23T01:48:52.455507+0000","last_unstale":"2024-04-23T01:49:46.344658+0000","last_undegraded":"2024-04-23T01:49:46.344658+0000","last_fullsized":"2024-04-23T01:49:46.344658+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":456,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.430351+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:48:53.430351+0000","last_peered":"2024-04-23T01:48:53.430351+0000","last_clean":"2024-04-23T01:48:53.430351+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:48:53.430351+0000","last_undegraded":"2024-04-23T01:48:53.430351+0000","last_fullsized":"2024-04-23T01:48:53.430351+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":65249280,"num_objects":997,"num_object_clones":0,"num_object_copies":2991,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7965,"num_write_kb":63720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97882112,"data_stored":195764224,"data_compressed":10105101,"data_compressed_allocated":97882112,"data_compressed_original":195764224,"omap_allocated":0,"internal_metadata":0},"log_size":7949,"ondisk_log_size":7949,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705943,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":93616,"kb_used_data":22376,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94278224,"statfs":{"total":96636764160,"available":96540901376,"internally_reserved":0,"allocated":22913024,"data_stored":45784050,"data_compressed":2357325,"data_compressed_allocated":22585344,"data_compressed_original":45662208,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705943,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":81204,"kb_used_data":28012,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94290636,"statfs":{"total":96636764160,"available":96553611264,"internally_reserved":0,"allocated":28684288,"data_stored":57326578,"data_compressed":2953202,"data_compressed_allocated":28356608,"data_compressed_original":57204736,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705943,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":81376,"kb_used_data":19864,"kb_used_omap":0,"kb_used_meta":61504,"kb_avail":94290464,"statfs":{"total":96636764160,"available":96553435136,"internally_reserved":0,"allocated":20340736,"data_stored":40171986,"data_compressed":2068187,"data_compressed_allocated":20033536,"data_compressed_original":40067072,"omap_allocated":0,"internal_metadata":62980096},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705943,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":82920,"kb_used_data":25568,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94288920,"statfs":{"total":96636764160,"available":96551854080,"internally_reserved":0,"allocated":26181632,"data_stored":52321266,"data_compressed":2694845,"data_compressed_allocated":25853952,"data_compressed_original":52199424,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26157056,"data_stored":52314112,"data_compressed":2700424,"data_compressed_allocated":26157056,"data_compressed_original":52314112,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20291584,"data_stored":40583168,"data_compressed":2094836,"data_compressed_allocated":20291584,"data_compressed_original":40583168,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28688384,"data_stored":57376768,"data_compressed":2961742,"data_compressed_allocated":28688384,"data_compressed_original":57376768,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":22745088,"data_stored":45490176,"data_compressed":2348099,"data_compressed_allocated":22745088,"data_compressed_original":45490176,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:49:53.408 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:49:53.408837+0000 min lat: 0.014864 max lat: 0.332875 avg lat: 0.108004 2024-04-23T01:49:53.408 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:49:53.408 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 60 16 8895 8879 1.15602 1.03906 0.130142 0.108004 2024-04-23T01:49:53.856 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:49:53.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:53.977+0000 7fe7a946d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:53.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:53.984+0000 7fe7a946d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:54.408 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 61 16 9042 9026 1.1559 1.14844 0.0791951 0.108032 2024-04-23T01:49:55.408 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 62 16 9182 9166 1.15489 1.09375 0.166308 0.108116 2024-04-23T01:49:56.252 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:49:56.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:56.367+0000 7faf5cb9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:56.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:56.373+0000 7faf5cb9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:56.408 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 63 16 9335 9319 1.15553 1.19531 0.157004 0.108032 2024-04-23T01:49:57.408 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 64 16 9472 9456 1.1542 1.07031 0.037692 0.108198 2024-04-23T01:49:57.646 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:49:57.646 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:49:57.646 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:49:57.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:57.786+0000 7fac1f15d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:57.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:57.796+0000 7fac1f15d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:57.999 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:49:58.000 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:49:58.013 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":175,"stamp":"2024-04-23T01:49:57.604394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":77169184,"num_objects":1172,"num_object_clones":0,"num_object_copies":3516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":9485,"num_write_kb":76678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9429,"ondisk_log_size":9429,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":334924,"kb_used_data":112428,"kb_used_omap":0,"kb_used_meta":222464,"kb_avail":377152436,"statfs":{"total":386547056640,"available":386204094464,"internally_reserved":0,"allocated":115126272,"data_stored":229617064,"data_compressed":11829855,"data_compressed_allocated":113836032,"data_compressed_original":229146624,"omap_allocated":0,"internal_metadata":227803136},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17121280,"num_objects":262,"num_object_clones":0,"num_object_copies":786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2090,"num_write_kb":16720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002287"},"pg_stats":[{"pgid":"5.f","version":"36'575","reported_seq":588,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:55.236477+0000","last_change":"2024-04-23T01:48:52.530638+0000","last_active":"2024-04-23T01:49:55.236477+0000","last_peered":"2024-04-23T01:49:55.236477+0000","last_clean":"2024-04-23T01:49:55.236477+0000","last_became_active":"2024-04-23T01:48:52.520759+0000","last_became_peered":"2024-04-23T01:48:52.520759+0000","last_unstale":"2024-04-23T01:49:55.236477+0000","last_undegraded":"2024-04-23T01:49:55.236477+0000","last_fullsized":"2024-04-23T01:49:55.236477+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718592,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":576,"num_write_kb":4608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"36'672","reported_seq":685,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:56.060413+0000","last_change":"2024-04-23T01:48:52.532479+0000","last_active":"2024-04-23T01:49:56.060413+0000","last_peered":"2024-04-23T01:49:56.060413+0000","last_clean":"2024-04-23T01:49:56.060413+0000","last_became_active":"2024-04-23T01:48:52.525595+0000","last_became_peered":"2024-04-23T01:48:52.525595+0000","last_unstale":"2024-04-23T01:49:56.060413+0000","last_undegraded":"2024-04-23T01:49:56.060413+0000","last_fullsized":"2024-04-23T01:49:56.060413+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5513216,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":673,"num_write_kb":5384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"36'695","reported_seq":708,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:54.991769+0000","last_change":"2024-04-23T01:48:52.571199+0000","last_active":"2024-04-23T01:49:54.991769+0000","last_peered":"2024-04-23T01:49:54.991769+0000","last_clean":"2024-04-23T01:49:54.991769+0000","last_became_active":"2024-04-23T01:48:52.566277+0000","last_became_peered":"2024-04-23T01:48:52.566277+0000","last_unstale":"2024-04-23T01:49:54.991769+0000","last_undegraded":"2024-04-23T01:49:54.991769+0000","last_fullsized":"2024-04-23T01:49:54.991769+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":696,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"36'573","reported_seq":586,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:56.583176+0000","last_change":"2024-04-23T01:48:52.497141+0000","last_active":"2024-04-23T01:49:56.583176+0000","last_peered":"2024-04-23T01:49:56.583176+0000","last_clean":"2024-04-23T01:49:56.583176+0000","last_became_active":"2024-04-23T01:48:52.489329+0000","last_became_peered":"2024-04-23T01:48:52.489329+0000","last_unstale":"2024-04-23T01:49:56.583176+0000","last_undegraded":"2024-04-23T01:49:56.583176+0000","last_fullsized":"2024-04-23T01:49:56.583176+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4702208,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":574,"num_write_kb":4592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"36'543","reported_seq":556,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:56.244668+0000","last_change":"2024-04-23T01:48:52.534975+0000","last_active":"2024-04-23T01:49:56.244668+0000","last_peered":"2024-04-23T01:49:56.244668+0000","last_clean":"2024-04-23T01:49:56.244668+0000","last_became_active":"2024-04-23T01:48:52.530664+0000","last_became_peered":"2024-04-23T01:48:52.530664+0000","last_unstale":"2024-04-23T01:49:56.244668+0000","last_undegraded":"2024-04-23T01:49:56.244668+0000","last_fullsized":"2024-04-23T01:49:56.244668+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":544,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"36'519","reported_seq":532,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:55.161400+0000","last_change":"2024-04-23T01:48:52.451606+0000","last_active":"2024-04-23T01:49:55.161400+0000","last_peered":"2024-04-23T01:49:55.161400+0000","last_clean":"2024-04-23T01:49:55.161400+0000","last_became_active":"2024-04-23T01:48:52.439020+0000","last_became_peered":"2024-04-23T01:48:52.439020+0000","last_unstale":"2024-04-23T01:49:55.161400+0000","last_undegraded":"2024-04-23T01:49:55.161400+0000","last_fullsized":"2024-04-23T01:49:55.161400+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259840,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":520,"num_write_kb":4160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"36'503","reported_seq":516,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:56.541359+0000","last_change":"2024-04-23T01:48:52.481437+0000","last_active":"2024-04-23T01:49:56.541359+0000","last_peered":"2024-04-23T01:49:56.541359+0000","last_clean":"2024-04-23T01:49:56.541359+0000","last_became_active":"2024-04-23T01:48:52.473748+0000","last_became_peered":"2024-04-23T01:48:52.473748+0000","last_unstale":"2024-04-23T01:49:56.541359+0000","last_undegraded":"2024-04-23T01:49:56.541359+0000","last_fullsized":"2024-04-23T01:49:56.541359+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4128768,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":504,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"36'652","reported_seq":665,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:56.569302+0000","last_change":"2024-04-23T01:48:52.568521+0000","last_active":"2024-04-23T01:49:56.569302+0000","last_peered":"2024-04-23T01:49:56.569302+0000","last_clean":"2024-04-23T01:49:56.569302+0000","last_became_active":"2024-04-23T01:48:52.562008+0000","last_became_peered":"2024-04-23T01:48:52.562008+0000","last_unstale":"2024-04-23T01:49:56.569302+0000","last_undegraded":"2024-04-23T01:49:56.569302+0000","last_fullsized":"2024-04-23T01:49:56.569302+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5349376,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":653,"num_write_kb":5224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"36'535","reported_seq":548,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:55.321052+0000","last_change":"2024-04-23T01:48:52.491272+0000","last_active":"2024-04-23T01:49:55.321052+0000","last_peered":"2024-04-23T01:49:55.321052+0000","last_clean":"2024-04-23T01:49:55.321052+0000","last_became_active":"2024-04-23T01:48:52.482878+0000","last_became_peered":"2024-04-23T01:48:52.482878+0000","last_unstale":"2024-04-23T01:49:55.321052+0000","last_undegraded":"2024-04-23T01:49:55.321052+0000","last_fullsized":"2024-04-23T01:49:55.321052+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4390912,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":536,"num_write_kb":4288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":203,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.336526+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:48:53.336526+0000","last_peered":"2024-04-23T01:48:53.336526+0000","last_clean":"2024-04-23T01:48:53.336526+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:48:53.336526+0000","last_undegraded":"2024-04-23T01:48:53.336526+0000","last_fullsized":"2024-04-23T01:48:53.336526+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.441413+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:48:53.441413+0000","last_peered":"2024-04-23T01:48:53.441413+0000","last_clean":"2024-04-23T01:48:53.441413+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:48:53.441413+0000","last_undegraded":"2024-04-23T01:48:53.441413+0000","last_fullsized":"2024-04-23T01:48:53.441413+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"36'607","reported_seq":620,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:55.603307+0000","last_change":"2024-04-23T01:48:52.550172+0000","last_active":"2024-04-23T01:49:55.603307+0000","last_peered":"2024-04-23T01:49:55.603307+0000","last_clean":"2024-04-23T01:49:55.603307+0000","last_became_active":"2024-04-23T01:48:52.541104+0000","last_became_peered":"2024-04-23T01:48:52.541104+0000","last_unstale":"2024-04-23T01:49:55.603307+0000","last_undegraded":"2024-04-23T01:49:55.603307+0000","last_fullsized":"2024-04-23T01:49:55.603307+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4980736,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":608,"num_write_kb":4864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.571998+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:48:53.571998+0000","last_peered":"2024-04-23T01:48:53.571998+0000","last_clean":"2024-04-23T01:48:53.571998+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:48:53.571998+0000","last_undegraded":"2024-04-23T01:48:53.571998+0000","last_fullsized":"2024-04-23T01:48:53.571998+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"36'567","reported_seq":580,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:53.929505+0000","last_change":"2024-04-23T01:48:52.526429+0000","last_active":"2024-04-23T01:49:53.929505+0000","last_peered":"2024-04-23T01:49:53.929505+0000","last_clean":"2024-04-23T01:49:53.929505+0000","last_became_active":"2024-04-23T01:48:52.514040+0000","last_became_peered":"2024-04-23T01:48:52.514040+0000","last_unstale":"2024-04-23T01:49:53.929505+0000","last_undegraded":"2024-04-23T01:49:53.929505+0000","last_fullsized":"2024-04-23T01:49:53.929505+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":568,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.432658+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:48:53.432658+0000","last_peered":"2024-04-23T01:48:53.432658+0000","last_clean":"2024-04-23T01:48:53.432658+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:48:53.432658+0000","last_undegraded":"2024-04-23T01:48:53.432658+0000","last_fullsized":"2024-04-23T01:48:53.432658+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"36'743","reported_seq":756,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:56.336408+0000","last_change":"2024-04-23T01:48:52.469625+0000","last_active":"2024-04-23T01:49:56.336408+0000","last_peered":"2024-04-23T01:49:56.336408+0000","last_clean":"2024-04-23T01:49:56.336408+0000","last_became_active":"2024-04-23T01:48:52.456212+0000","last_became_peered":"2024-04-23T01:48:52.456212+0000","last_unstale":"2024-04-23T01:49:56.336408+0000","last_undegraded":"2024-04-23T01:49:56.336408+0000","last_fullsized":"2024-04-23T01:49:56.336408+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6094848,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":744,"num_write_kb":5952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":68,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.605161+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:48:53.605161+0000","last_peered":"2024-04-23T01:48:53.605161+0000","last_clean":"2024-04-23T01:48:53.605161+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:48:53.605161+0000","last_undegraded":"2024-04-23T01:48:53.605161+0000","last_fullsized":"2024-04-23T01:48:53.605161+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"36'535","reported_seq":548,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:55.743143+0000","last_change":"2024-04-23T01:48:52.505215+0000","last_active":"2024-04-23T01:49:55.743143+0000","last_peered":"2024-04-23T01:49:55.743143+0000","last_clean":"2024-04-23T01:49:55.743143+0000","last_became_active":"2024-04-23T01:48:52.497519+0000","last_became_peered":"2024-04-23T01:48:52.497519+0000","last_unstale":"2024-04-23T01:49:55.743143+0000","last_undegraded":"2024-04-23T01:49:55.743143+0000","last_fullsized":"2024-04-23T01:49:55.743143+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4390912,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":536,"num_write_kb":4288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.438431+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:48:53.438431+0000","last_peered":"2024-04-23T01:48:53.438431+0000","last_clean":"2024-04-23T01:48:53.438431+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:48:53.438431+0000","last_undegraded":"2024-04-23T01:48:53.438431+0000","last_fullsized":"2024-04-23T01:48:53.438431+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"36'599","reported_seq":612,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:55.511182+0000","last_change":"2024-04-23T01:48:52.501817+0000","last_active":"2024-04-23T01:49:55.511182+0000","last_peered":"2024-04-23T01:49:55.511182+0000","last_clean":"2024-04-23T01:49:55.511182+0000","last_became_active":"2024-04-23T01:48:52.496130+0000","last_became_peered":"2024-04-23T01:48:52.496130+0000","last_unstale":"2024-04-23T01:49:55.511182+0000","last_undegraded":"2024-04-23T01:49:55.511182+0000","last_fullsized":"2024-04-23T01:49:55.511182+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.346665+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:48:53.346665+0000","last_peered":"2024-04-23T01:48:53.346665+0000","last_clean":"2024-04-23T01:48:53.346665+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:48:53.346665+0000","last_undegraded":"2024-04-23T01:48:53.346665+0000","last_fullsized":"2024-04-23T01:48:53.346665+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"36'527","reported_seq":540,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:56.377221+0000","last_change":"2024-04-23T01:48:52.473323+0000","last_active":"2024-04-23T01:49:56.377221+0000","last_peered":"2024-04-23T01:49:56.377221+0000","last_clean":"2024-04-23T01:49:56.377221+0000","last_became_active":"2024-04-23T01:48:52.466602+0000","last_became_peered":"2024-04-23T01:48:52.466602+0000","last_unstale":"2024-04-23T01:49:56.377221+0000","last_undegraded":"2024-04-23T01:49:56.377221+0000","last_fullsized":"2024-04-23T01:49:56.377221+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":528,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.439757+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:48:53.439757+0000","last_peered":"2024-04-23T01:48:53.439757+0000","last_clean":"2024-04-23T01:48:53.439757+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:48:53.439757+0000","last_undegraded":"2024-04-23T01:48:53.439757+0000","last_fullsized":"2024-04-23T01:48:53.439757+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"36'487","reported_seq":500,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:56.420468+0000","last_change":"2024-04-23T01:48:52.464660+0000","last_active":"2024-04-23T01:49:56.420468+0000","last_peered":"2024-04-23T01:49:56.420468+0000","last_clean":"2024-04-23T01:49:56.420468+0000","last_became_active":"2024-04-23T01:48:52.455507+0000","last_became_peered":"2024-04-23T01:48:52.455507+0000","last_unstale":"2024-04-23T01:49:56.420468+0000","last_undegraded":"2024-04-23T01:49:56.420468+0000","last_fullsized":"2024-04-23T01:49:56.420468+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3997696,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":488,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.430351+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:48:53.430351+0000","last_peered":"2024-04-23T01:48:53.430351+0000","last_clean":"2024-04-23T01:48:53.430351+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:48:53.430351+0000","last_undegraded":"2024-04-23T01:48:53.430351+0000","last_fullsized":"2024-04-23T01:48:53.430351+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":76578816,"num_objects":1170,"num_object_clones":0,"num_object_copies":3510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9348,"num_write_kb":74784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114864128,"data_stored":229728256,"data_compressed":11858859,"data_compressed_allocated":114864128,"data_compressed_original":229728256,"omap_allocated":0,"internal_metadata":0},"log_size":9332,"ondisk_log_size":9332,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":101696,"kb_used_data":26296,"kb_used_omap":0,"kb_used_meta":75392,"kb_avail":94270144,"statfs":{"total":96636764160,"available":96532627456,"internally_reserved":0,"allocated":26927104,"data_stored":53812210,"data_compressed":2771865,"data_compressed_allocated":26599424,"data_compressed_original":53690368,"omap_allocated":0,"internal_metadata":77201408},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":90192,"kb_used_data":32840,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94281648,"statfs":{"total":96636764160,"available":96544407552,"internally_reserved":0,"allocated":33628160,"data_stored":67214322,"data_compressed":3463763,"data_compressed_allocated":33300480,"data_compressed_original":67092480,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":68064,"kb_used_data":23192,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94303776,"statfs":{"total":96636764160,"available":96567066624,"internally_reserved":0,"allocated":23748608,"data_stored":46987730,"data_compressed":2420123,"data_compressed_allocated":23441408,"data_compressed_original":46882816,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":74972,"kb_used_data":30100,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94296868,"statfs":{"total":96636764160,"available":96559992832,"internally_reserved":0,"allocated":30822400,"data_stored":61602802,"data_compressed":3174104,"data_compressed_allocated":30494720,"data_compressed_original":61480960,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":30695424,"data_stored":61390848,"data_compressed":3169108,"data_compressed_allocated":30695424,"data_compressed_original":61390848,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":23711744,"data_stored":47423488,"data_compressed":2448041,"data_compressed_allocated":23711744,"data_compressed_original":47423488,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":33595392,"data_stored":67190784,"data_compressed":3468496,"data_compressed_allocated":33595392,"data_compressed_original":67190784,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":26861568,"data_stored":53723136,"data_compressed":2773214,"data_compressed_allocated":26861568,"data_compressed_original":53723136,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:49:58.408 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 65 16 9617 9601 1.15387 1.13281 0.208315 0.108178 2024-04-23T01:49:58.539 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:49:58.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:58.670+0000 7f702211d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:58.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:49:58.677+0000 7f702211d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:49:59.408 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 66 16 9747 9731 1.15177 1.01562 0.218417 0.108393 2024-04-23T01:50:00.408 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 67 16 9882 9866 1.15032 1.05469 0.256998 0.108549 2024-04-23T01:50:00.835 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:50:00.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:00.971+0000 7f6d65b5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:00.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:00.978+0000 7f6d65b5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:01.408 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 68 16 10014 9998 1.14857 1.03125 0.0287048 0.108683 2024-04-23T01:50:02.408 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 69 16 10149 10133 1.14721 1.05469 0.0540789 0.108836 2024-04-23T01:50:03.014 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:50:03.015 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:50:03.015 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:50:03.126 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:50:03.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:03.135+0000 7f6a472ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:03.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:03.144+0000 7f6a472ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:03.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:03.260+0000 7f2dd06ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:03.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:03.266+0000 7f2dd06ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:03.393 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:50:03.393 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:50:03.408 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 70 16 10288 10272 1.14633 1.08594 0.105155 0.108903 2024-04-23T01:50:03.410 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":177,"stamp":"2024-04-23T01:50:01.605150+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":77169184,"num_objects":1172,"num_object_clones":0,"num_object_copies":3516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":9485,"num_write_kb":76678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9429,"ondisk_log_size":9429,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":334924,"kb_used_data":112428,"kb_used_omap":0,"kb_used_meta":222464,"kb_avail":377152436,"statfs":{"total":386547056640,"available":386204094464,"internally_reserved":0,"allocated":115126272,"data_stored":229617064,"data_compressed":11829855,"data_compressed_allocated":113836032,"data_compressed_original":229146624,"omap_allocated":0,"internal_metadata":227803136},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11329536,"num_objects":173,"num_object_clones":0,"num_object_copies":519,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1383,"num_write_kb":11064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002248"},"pg_stats":[{"pgid":"5.f","version":"36'575","reported_seq":588,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:55.236477+0000","last_change":"2024-04-23T01:48:52.530638+0000","last_active":"2024-04-23T01:49:55.236477+0000","last_peered":"2024-04-23T01:49:55.236477+0000","last_clean":"2024-04-23T01:49:55.236477+0000","last_became_active":"2024-04-23T01:48:52.520759+0000","last_became_peered":"2024-04-23T01:48:52.520759+0000","last_unstale":"2024-04-23T01:49:55.236477+0000","last_undegraded":"2024-04-23T01:49:55.236477+0000","last_fullsized":"2024-04-23T01:49:55.236477+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718592,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":576,"num_write_kb":4608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"36'672","reported_seq":685,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:56.060413+0000","last_change":"2024-04-23T01:48:52.532479+0000","last_active":"2024-04-23T01:49:56.060413+0000","last_peered":"2024-04-23T01:49:56.060413+0000","last_clean":"2024-04-23T01:49:56.060413+0000","last_became_active":"2024-04-23T01:48:52.525595+0000","last_became_peered":"2024-04-23T01:48:52.525595+0000","last_unstale":"2024-04-23T01:49:56.060413+0000","last_undegraded":"2024-04-23T01:49:56.060413+0000","last_fullsized":"2024-04-23T01:49:56.060413+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5513216,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":673,"num_write_kb":5384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"36'695","reported_seq":708,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:54.991769+0000","last_change":"2024-04-23T01:48:52.571199+0000","last_active":"2024-04-23T01:49:54.991769+0000","last_peered":"2024-04-23T01:49:54.991769+0000","last_clean":"2024-04-23T01:49:54.991769+0000","last_became_active":"2024-04-23T01:48:52.566277+0000","last_became_peered":"2024-04-23T01:48:52.566277+0000","last_unstale":"2024-04-23T01:49:54.991769+0000","last_undegraded":"2024-04-23T01:49:54.991769+0000","last_fullsized":"2024-04-23T01:49:54.991769+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":696,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"36'573","reported_seq":586,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:56.583176+0000","last_change":"2024-04-23T01:48:52.497141+0000","last_active":"2024-04-23T01:49:56.583176+0000","last_peered":"2024-04-23T01:49:56.583176+0000","last_clean":"2024-04-23T01:49:56.583176+0000","last_became_active":"2024-04-23T01:48:52.489329+0000","last_became_peered":"2024-04-23T01:48:52.489329+0000","last_unstale":"2024-04-23T01:49:56.583176+0000","last_undegraded":"2024-04-23T01:49:56.583176+0000","last_fullsized":"2024-04-23T01:49:56.583176+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4702208,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":574,"num_write_kb":4592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"36'543","reported_seq":556,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:56.244668+0000","last_change":"2024-04-23T01:48:52.534975+0000","last_active":"2024-04-23T01:49:56.244668+0000","last_peered":"2024-04-23T01:49:56.244668+0000","last_clean":"2024-04-23T01:49:56.244668+0000","last_became_active":"2024-04-23T01:48:52.530664+0000","last_became_peered":"2024-04-23T01:48:52.530664+0000","last_unstale":"2024-04-23T01:49:56.244668+0000","last_undegraded":"2024-04-23T01:49:56.244668+0000","last_fullsized":"2024-04-23T01:49:56.244668+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":544,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"36'519","reported_seq":532,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:55.161400+0000","last_change":"2024-04-23T01:48:52.451606+0000","last_active":"2024-04-23T01:49:55.161400+0000","last_peered":"2024-04-23T01:49:55.161400+0000","last_clean":"2024-04-23T01:49:55.161400+0000","last_became_active":"2024-04-23T01:48:52.439020+0000","last_became_peered":"2024-04-23T01:48:52.439020+0000","last_unstale":"2024-04-23T01:49:55.161400+0000","last_undegraded":"2024-04-23T01:49:55.161400+0000","last_fullsized":"2024-04-23T01:49:55.161400+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259840,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":520,"num_write_kb":4160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"36'503","reported_seq":516,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:56.541359+0000","last_change":"2024-04-23T01:48:52.481437+0000","last_active":"2024-04-23T01:49:56.541359+0000","last_peered":"2024-04-23T01:49:56.541359+0000","last_clean":"2024-04-23T01:49:56.541359+0000","last_became_active":"2024-04-23T01:48:52.473748+0000","last_became_peered":"2024-04-23T01:48:52.473748+0000","last_unstale":"2024-04-23T01:49:56.541359+0000","last_undegraded":"2024-04-23T01:49:56.541359+0000","last_fullsized":"2024-04-23T01:49:56.541359+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4128768,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":504,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"36'652","reported_seq":665,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:56.569302+0000","last_change":"2024-04-23T01:48:52.568521+0000","last_active":"2024-04-23T01:49:56.569302+0000","last_peered":"2024-04-23T01:49:56.569302+0000","last_clean":"2024-04-23T01:49:56.569302+0000","last_became_active":"2024-04-23T01:48:52.562008+0000","last_became_peered":"2024-04-23T01:48:52.562008+0000","last_unstale":"2024-04-23T01:49:56.569302+0000","last_undegraded":"2024-04-23T01:49:56.569302+0000","last_fullsized":"2024-04-23T01:49:56.569302+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5349376,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":653,"num_write_kb":5224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"36'535","reported_seq":548,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:55.321052+0000","last_change":"2024-04-23T01:48:52.491272+0000","last_active":"2024-04-23T01:49:55.321052+0000","last_peered":"2024-04-23T01:49:55.321052+0000","last_clean":"2024-04-23T01:49:55.321052+0000","last_became_active":"2024-04-23T01:48:52.482878+0000","last_became_peered":"2024-04-23T01:48:52.482878+0000","last_unstale":"2024-04-23T01:49:55.321052+0000","last_undegraded":"2024-04-23T01:49:55.321052+0000","last_fullsized":"2024-04-23T01:49:55.321052+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4390912,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":536,"num_write_kb":4288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":203,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.336526+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:48:53.336526+0000","last_peered":"2024-04-23T01:48:53.336526+0000","last_clean":"2024-04-23T01:48:53.336526+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:48:53.336526+0000","last_undegraded":"2024-04-23T01:48:53.336526+0000","last_fullsized":"2024-04-23T01:48:53.336526+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.441413+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:48:53.441413+0000","last_peered":"2024-04-23T01:48:53.441413+0000","last_clean":"2024-04-23T01:48:53.441413+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:48:53.441413+0000","last_undegraded":"2024-04-23T01:48:53.441413+0000","last_fullsized":"2024-04-23T01:48:53.441413+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"36'607","reported_seq":620,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:55.603307+0000","last_change":"2024-04-23T01:48:52.550172+0000","last_active":"2024-04-23T01:49:55.603307+0000","last_peered":"2024-04-23T01:49:55.603307+0000","last_clean":"2024-04-23T01:49:55.603307+0000","last_became_active":"2024-04-23T01:48:52.541104+0000","last_became_peered":"2024-04-23T01:48:52.541104+0000","last_unstale":"2024-04-23T01:49:55.603307+0000","last_undegraded":"2024-04-23T01:49:55.603307+0000","last_fullsized":"2024-04-23T01:49:55.603307+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4980736,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":608,"num_write_kb":4864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.571998+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:48:53.571998+0000","last_peered":"2024-04-23T01:48:53.571998+0000","last_clean":"2024-04-23T01:48:53.571998+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:48:53.571998+0000","last_undegraded":"2024-04-23T01:48:53.571998+0000","last_fullsized":"2024-04-23T01:48:53.571998+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"36'567","reported_seq":580,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:53.929505+0000","last_change":"2024-04-23T01:48:52.526429+0000","last_active":"2024-04-23T01:49:53.929505+0000","last_peered":"2024-04-23T01:49:53.929505+0000","last_clean":"2024-04-23T01:49:53.929505+0000","last_became_active":"2024-04-23T01:48:52.514040+0000","last_became_peered":"2024-04-23T01:48:52.514040+0000","last_unstale":"2024-04-23T01:49:53.929505+0000","last_undegraded":"2024-04-23T01:49:53.929505+0000","last_fullsized":"2024-04-23T01:49:53.929505+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":568,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.432658+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:48:53.432658+0000","last_peered":"2024-04-23T01:48:53.432658+0000","last_clean":"2024-04-23T01:48:53.432658+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:48:53.432658+0000","last_undegraded":"2024-04-23T01:48:53.432658+0000","last_fullsized":"2024-04-23T01:48:53.432658+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"36'743","reported_seq":756,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:56.336408+0000","last_change":"2024-04-23T01:48:52.469625+0000","last_active":"2024-04-23T01:49:56.336408+0000","last_peered":"2024-04-23T01:49:56.336408+0000","last_clean":"2024-04-23T01:49:56.336408+0000","last_became_active":"2024-04-23T01:48:52.456212+0000","last_became_peered":"2024-04-23T01:48:52.456212+0000","last_unstale":"2024-04-23T01:49:56.336408+0000","last_undegraded":"2024-04-23T01:49:56.336408+0000","last_fullsized":"2024-04-23T01:49:56.336408+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6094848,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":744,"num_write_kb":5952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":68,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.605161+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:48:53.605161+0000","last_peered":"2024-04-23T01:48:53.605161+0000","last_clean":"2024-04-23T01:48:53.605161+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:48:53.605161+0000","last_undegraded":"2024-04-23T01:48:53.605161+0000","last_fullsized":"2024-04-23T01:48:53.605161+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"36'535","reported_seq":548,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:55.743143+0000","last_change":"2024-04-23T01:48:52.505215+0000","last_active":"2024-04-23T01:49:55.743143+0000","last_peered":"2024-04-23T01:49:55.743143+0000","last_clean":"2024-04-23T01:49:55.743143+0000","last_became_active":"2024-04-23T01:48:52.497519+0000","last_became_peered":"2024-04-23T01:48:52.497519+0000","last_unstale":"2024-04-23T01:49:55.743143+0000","last_undegraded":"2024-04-23T01:49:55.743143+0000","last_fullsized":"2024-04-23T01:49:55.743143+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4390912,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":536,"num_write_kb":4288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.438431+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:48:53.438431+0000","last_peered":"2024-04-23T01:48:53.438431+0000","last_clean":"2024-04-23T01:48:53.438431+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:48:53.438431+0000","last_undegraded":"2024-04-23T01:48:53.438431+0000","last_fullsized":"2024-04-23T01:48:53.438431+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"36'599","reported_seq":612,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:55.511182+0000","last_change":"2024-04-23T01:48:52.501817+0000","last_active":"2024-04-23T01:49:55.511182+0000","last_peered":"2024-04-23T01:49:55.511182+0000","last_clean":"2024-04-23T01:49:55.511182+0000","last_became_active":"2024-04-23T01:48:52.496130+0000","last_became_peered":"2024-04-23T01:48:52.496130+0000","last_unstale":"2024-04-23T01:49:55.511182+0000","last_undegraded":"2024-04-23T01:49:55.511182+0000","last_fullsized":"2024-04-23T01:49:55.511182+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.346665+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:48:53.346665+0000","last_peered":"2024-04-23T01:48:53.346665+0000","last_clean":"2024-04-23T01:48:53.346665+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:48:53.346665+0000","last_undegraded":"2024-04-23T01:48:53.346665+0000","last_fullsized":"2024-04-23T01:48:53.346665+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"36'527","reported_seq":540,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:56.377221+0000","last_change":"2024-04-23T01:48:52.473323+0000","last_active":"2024-04-23T01:49:56.377221+0000","last_peered":"2024-04-23T01:49:56.377221+0000","last_clean":"2024-04-23T01:49:56.377221+0000","last_became_active":"2024-04-23T01:48:52.466602+0000","last_became_peered":"2024-04-23T01:48:52.466602+0000","last_unstale":"2024-04-23T01:49:56.377221+0000","last_undegraded":"2024-04-23T01:49:56.377221+0000","last_fullsized":"2024-04-23T01:49:56.377221+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":528,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.439757+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:48:53.439757+0000","last_peered":"2024-04-23T01:48:53.439757+0000","last_clean":"2024-04-23T01:48:53.439757+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:48:53.439757+0000","last_undegraded":"2024-04-23T01:48:53.439757+0000","last_fullsized":"2024-04-23T01:48:53.439757+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"36'487","reported_seq":500,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:49:56.420468+0000","last_change":"2024-04-23T01:48:52.464660+0000","last_active":"2024-04-23T01:49:56.420468+0000","last_peered":"2024-04-23T01:49:56.420468+0000","last_clean":"2024-04-23T01:49:56.420468+0000","last_became_active":"2024-04-23T01:48:52.455507+0000","last_became_peered":"2024-04-23T01:48:52.455507+0000","last_unstale":"2024-04-23T01:49:56.420468+0000","last_undegraded":"2024-04-23T01:49:56.420468+0000","last_fullsized":"2024-04-23T01:49:56.420468+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3997696,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":488,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.430351+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:48:53.430351+0000","last_peered":"2024-04-23T01:48:53.430351+0000","last_clean":"2024-04-23T01:48:53.430351+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:48:53.430351+0000","last_undegraded":"2024-04-23T01:48:53.430351+0000","last_fullsized":"2024-04-23T01:48:53.430351+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":76578816,"num_objects":1170,"num_object_clones":0,"num_object_copies":3510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9348,"num_write_kb":74784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114864128,"data_stored":229728256,"data_compressed":11858859,"data_compressed_allocated":114864128,"data_compressed_original":229728256,"omap_allocated":0,"internal_metadata":0},"log_size":9332,"ondisk_log_size":9332,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":101696,"kb_used_data":26296,"kb_used_omap":0,"kb_used_meta":75392,"kb_avail":94270144,"statfs":{"total":96636764160,"available":96532627456,"internally_reserved":0,"allocated":26927104,"data_stored":53812210,"data_compressed":2771865,"data_compressed_allocated":26599424,"data_compressed_original":53690368,"omap_allocated":0,"internal_metadata":77201408},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":90192,"kb_used_data":32840,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94281648,"statfs":{"total":96636764160,"available":96544407552,"internally_reserved":0,"allocated":33628160,"data_stored":67214322,"data_compressed":3463763,"data_compressed_allocated":33300480,"data_compressed_original":67092480,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":68064,"kb_used_data":23192,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94303776,"statfs":{"total":96636764160,"available":96567066624,"internally_reserved":0,"allocated":23748608,"data_stored":46987730,"data_compressed":2420123,"data_compressed_allocated":23441408,"data_compressed_original":46882816,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":74972,"kb_used_data":30100,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94296868,"statfs":{"total":96636764160,"available":96559992832,"internally_reserved":0,"allocated":30822400,"data_stored":61602802,"data_compressed":3174104,"data_compressed_allocated":30494720,"data_compressed_original":61480960,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":30695424,"data_stored":61390848,"data_compressed":3169108,"data_compressed_allocated":30695424,"data_compressed_original":61390848,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":23711744,"data_stored":47423488,"data_compressed":2448041,"data_compressed_allocated":23711744,"data_compressed_original":47423488,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":33595392,"data_stored":67190784,"data_compressed":3468496,"data_compressed_allocated":33595392,"data_compressed_original":67190784,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":26861568,"data_stored":53723136,"data_compressed":2773214,"data_compressed_allocated":26861568,"data_compressed_original":53723136,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:50:04.409 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 71 16 10421 10405 1.14482 1.03906 0.0816021 0.109056 2024-04-23T01:50:05.409 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 72 16 10561 10545 1.14411 1.09375 0.0382213 0.109156 2024-04-23T01:50:05.559 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:50:05.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:05.675+0000 7f4df502d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:05.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:05.682+0000 7f4df502d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:06.409 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 73 16 10688 10672 1.14203 0.992188 0.14767 0.109336 2024-04-23T01:50:07.409 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 74 16 10834 10818 1.14201 1.14062 0.133173 0.109367 2024-04-23T01:50:07.849 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:50:07.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:07.995+0000 7f29e26ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:08.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:08.001+0000 7f29e26ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:08.409 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 75 16 10960 10944 1.1399 0.984375 0.120908 0.109575 2024-04-23T01:50:08.412 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:50:08.412 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:50:08.413 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:50:08.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:08.545+0000 7f0ec8e2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:08.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:08.556+0000 7f0ec8e2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:08.818 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:50:08.818 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:50:08.839 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2024-04-23T01:50:07.606261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":88261152,"num_objects":1342,"num_object_clones":0,"num_object_copies":4026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":10839,"num_write_kb":87510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10783,"ondisk_log_size":10783,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":351244,"kb_used_data":128748,"kb_used_omap":0,"kb_used_meta":222464,"kb_avail":377136116,"statfs":{"total":386547056640,"available":386187382784,"internally_reserved":0,"allocated":131837952,"data_stored":263040424,"data_compressed":13555695,"data_compressed_allocated":130547712,"data_compressed_original":262569984,"omap_allocated":0,"internal_metadata":227803136},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16916480,"num_objects":259,"num_object_clones":0,"num_object_copies":777,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2065,"num_write_kb":16520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002177"},"pg_stats":[{"pgid":"5.f","version":"36'631","reported_seq":644,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:04.276850+0000","last_change":"2024-04-23T01:48:52.530638+0000","last_active":"2024-04-23T01:50:04.276850+0000","last_peered":"2024-04-23T01:50:04.276850+0000","last_clean":"2024-04-23T01:50:04.276850+0000","last_became_active":"2024-04-23T01:48:52.520759+0000","last_became_peered":"2024-04-23T01:48:52.520759+0000","last_unstale":"2024-04-23T01:50:04.276850+0000","last_undegraded":"2024-04-23T01:50:04.276850+0000","last_fullsized":"2024-04-23T01:50:04.276850+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":5056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"36'807","reported_seq":820,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:06.344980+0000","last_change":"2024-04-23T01:48:52.532479+0000","last_active":"2024-04-23T01:50:06.344980+0000","last_peered":"2024-04-23T01:50:06.344980+0000","last_clean":"2024-04-23T01:50:06.344980+0000","last_became_active":"2024-04-23T01:48:52.525595+0000","last_became_peered":"2024-04-23T01:48:52.525595+0000","last_unstale":"2024-04-23T01:50:06.344980+0000","last_undegraded":"2024-04-23T01:50:06.344980+0000","last_fullsized":"2024-04-23T01:50:06.344980+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6619136,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":808,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"36'751","reported_seq":764,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:04.069108+0000","last_change":"2024-04-23T01:48:52.571199+0000","last_active":"2024-04-23T01:50:04.069108+0000","last_peered":"2024-04-23T01:50:04.069108+0000","last_clean":"2024-04-23T01:50:04.069108+0000","last_became_active":"2024-04-23T01:48:52.566277+0000","last_became_peered":"2024-04-23T01:48:52.566277+0000","last_unstale":"2024-04-23T01:50:04.069108+0000","last_undegraded":"2024-04-23T01:50:04.069108+0000","last_fullsized":"2024-04-23T01:50:04.069108+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"36'699","reported_seq":712,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:06.586825+0000","last_change":"2024-04-23T01:48:52.497141+0000","last_active":"2024-04-23T01:50:06.586825+0000","last_peered":"2024-04-23T01:50:06.586825+0000","last_clean":"2024-04-23T01:50:06.586825+0000","last_became_active":"2024-04-23T01:48:52.489329+0000","last_became_peered":"2024-04-23T01:48:52.489329+0000","last_unstale":"2024-04-23T01:50:06.586825+0000","last_undegraded":"2024-04-23T01:50:06.586825+0000","last_fullsized":"2024-04-23T01:50:06.586825+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5734400,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":700,"num_write_kb":5600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"36'607","reported_seq":620,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:05.343182+0000","last_change":"2024-04-23T01:48:52.534975+0000","last_active":"2024-04-23T01:50:05.343182+0000","last_peered":"2024-04-23T01:50:05.343182+0000","last_clean":"2024-04-23T01:50:05.343182+0000","last_became_active":"2024-04-23T01:48:52.530664+0000","last_became_peered":"2024-04-23T01:48:52.530664+0000","last_unstale":"2024-04-23T01:50:05.343182+0000","last_undegraded":"2024-04-23T01:50:05.343182+0000","last_fullsized":"2024-04-23T01:50:05.343182+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4980736,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":608,"num_write_kb":4864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"36'591","reported_seq":604,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:06.022318+0000","last_change":"2024-04-23T01:48:52.451606+0000","last_active":"2024-04-23T01:50:06.022318+0000","last_peered":"2024-04-23T01:50:06.022318+0000","last_clean":"2024-04-23T01:50:06.022318+0000","last_became_active":"2024-04-23T01:48:52.439020+0000","last_became_peered":"2024-04-23T01:48:52.439020+0000","last_unstale":"2024-04-23T01:50:06.022318+0000","last_undegraded":"2024-04-23T01:50:06.022318+0000","last_fullsized":"2024-04-23T01:50:06.022318+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":592,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"36'599","reported_seq":612,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:06.258614+0000","last_change":"2024-04-23T01:48:52.481437+0000","last_active":"2024-04-23T01:50:06.258614+0000","last_peered":"2024-04-23T01:50:06.258614+0000","last_clean":"2024-04-23T01:50:06.258614+0000","last_became_active":"2024-04-23T01:48:52.473748+0000","last_became_peered":"2024-04-23T01:48:52.473748+0000","last_unstale":"2024-04-23T01:50:06.258614+0000","last_undegraded":"2024-04-23T01:50:06.258614+0000","last_fullsized":"2024-04-23T01:50:06.258614+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"36'719","reported_seq":732,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:06.444301+0000","last_change":"2024-04-23T01:48:52.568521+0000","last_active":"2024-04-23T01:50:06.444301+0000","last_peered":"2024-04-23T01:50:06.444301+0000","last_clean":"2024-04-23T01:50:06.444301+0000","last_became_active":"2024-04-23T01:48:52.562008+0000","last_became_peered":"2024-04-23T01:48:52.562008+0000","last_unstale":"2024-04-23T01:50:06.444301+0000","last_undegraded":"2024-04-23T01:50:06.444301+0000","last_fullsized":"2024-04-23T01:50:06.444301+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":720,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"36'672","reported_seq":685,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:06.598431+0000","last_change":"2024-04-23T01:48:52.491272+0000","last_active":"2024-04-23T01:50:06.598431+0000","last_peered":"2024-04-23T01:50:06.598431+0000","last_clean":"2024-04-23T01:50:06.598431+0000","last_became_active":"2024-04-23T01:48:52.482878+0000","last_became_peered":"2024-04-23T01:48:52.482878+0000","last_unstale":"2024-04-23T01:50:06.598431+0000","last_undegraded":"2024-04-23T01:50:06.598431+0000","last_fullsized":"2024-04-23T01:50:06.598431+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5513216,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":673,"num_write_kb":5384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":203,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.336526+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:48:53.336526+0000","last_peered":"2024-04-23T01:48:53.336526+0000","last_clean":"2024-04-23T01:48:53.336526+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:48:53.336526+0000","last_undegraded":"2024-04-23T01:48:53.336526+0000","last_fullsized":"2024-04-23T01:48:53.336526+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.441413+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:48:53.441413+0000","last_peered":"2024-04-23T01:48:53.441413+0000","last_clean":"2024-04-23T01:48:53.441413+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:48:53.441413+0000","last_undegraded":"2024-04-23T01:48:53.441413+0000","last_fullsized":"2024-04-23T01:48:53.441413+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"36'712","reported_seq":725,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:06.553961+0000","last_change":"2024-04-23T01:48:52.550172+0000","last_active":"2024-04-23T01:50:06.553961+0000","last_peered":"2024-04-23T01:50:06.553961+0000","last_clean":"2024-04-23T01:50:06.553961+0000","last_became_active":"2024-04-23T01:48:52.541104+0000","last_became_peered":"2024-04-23T01:48:52.541104+0000","last_unstale":"2024-04-23T01:50:06.553961+0000","last_undegraded":"2024-04-23T01:50:06.553961+0000","last_fullsized":"2024-04-23T01:50:06.553961+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5840896,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":713,"num_write_kb":5704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.571998+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:48:53.571998+0000","last_peered":"2024-04-23T01:48:53.571998+0000","last_clean":"2024-04-23T01:48:53.571998+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:48:53.571998+0000","last_undegraded":"2024-04-23T01:48:53.571998+0000","last_fullsized":"2024-04-23T01:48:53.571998+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"36'639","reported_seq":652,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:04.674807+0000","last_change":"2024-04-23T01:48:52.526429+0000","last_active":"2024-04-23T01:50:04.674807+0000","last_peered":"2024-04-23T01:50:04.674807+0000","last_clean":"2024-04-23T01:50:04.674807+0000","last_became_active":"2024-04-23T01:48:52.514040+0000","last_became_peered":"2024-04-23T01:48:52.514040+0000","last_unstale":"2024-04-23T01:50:04.674807+0000","last_undegraded":"2024-04-23T01:50:04.674807+0000","last_fullsized":"2024-04-23T01:50:04.674807+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":640,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.432658+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:48:53.432658+0000","last_peered":"2024-04-23T01:48:53.432658+0000","last_clean":"2024-04-23T01:48:53.432658+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:48:53.432658+0000","last_undegraded":"2024-04-23T01:48:53.432658+0000","last_fullsized":"2024-04-23T01:48:53.432658+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"36'831","reported_seq":844,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:05.316328+0000","last_change":"2024-04-23T01:48:52.469625+0000","last_active":"2024-04-23T01:50:05.316328+0000","last_peered":"2024-04-23T01:50:05.316328+0000","last_clean":"2024-04-23T01:50:05.316328+0000","last_became_active":"2024-04-23T01:48:52.456212+0000","last_became_peered":"2024-04-23T01:48:52.456212+0000","last_unstale":"2024-04-23T01:50:05.316328+0000","last_undegraded":"2024-04-23T01:50:05.316328+0000","last_fullsized":"2024-04-23T01:50:05.316328+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":831,"log_dups_size":0,"ondisk_log_size":831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6815744,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":68,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.605161+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:48:53.605161+0000","last_peered":"2024-04-23T01:48:53.605161+0000","last_clean":"2024-04-23T01:48:53.605161+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:48:53.605161+0000","last_undegraded":"2024-04-23T01:48:53.605161+0000","last_fullsized":"2024-04-23T01:48:53.605161+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"36'615","reported_seq":628,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:05.844693+0000","last_change":"2024-04-23T01:48:52.505215+0000","last_active":"2024-04-23T01:50:05.844693+0000","last_peered":"2024-04-23T01:50:05.844693+0000","last_clean":"2024-04-23T01:50:05.844693+0000","last_became_active":"2024-04-23T01:48:52.497519+0000","last_became_peered":"2024-04-23T01:48:52.497519+0000","last_unstale":"2024-04-23T01:50:05.844693+0000","last_undegraded":"2024-04-23T01:50:05.844693+0000","last_fullsized":"2024-04-23T01:50:05.844693+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5046272,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":616,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.438431+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:48:53.438431+0000","last_peered":"2024-04-23T01:48:53.438431+0000","last_clean":"2024-04-23T01:48:53.438431+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:48:53.438431+0000","last_undegraded":"2024-04-23T01:48:53.438431+0000","last_fullsized":"2024-04-23T01:48:53.438431+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"36'639","reported_seq":652,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:05.608215+0000","last_change":"2024-04-23T01:48:52.501817+0000","last_active":"2024-04-23T01:50:05.608215+0000","last_peered":"2024-04-23T01:50:05.608215+0000","last_clean":"2024-04-23T01:50:05.608215+0000","last_became_active":"2024-04-23T01:48:52.496130+0000","last_became_peered":"2024-04-23T01:48:52.496130+0000","last_unstale":"2024-04-23T01:50:05.608215+0000","last_undegraded":"2024-04-23T01:50:05.608215+0000","last_fullsized":"2024-04-23T01:50:05.608215+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":640,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.346665+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:48:53.346665+0000","last_peered":"2024-04-23T01:48:53.346665+0000","last_clean":"2024-04-23T01:48:53.346665+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:48:53.346665+0000","last_undegraded":"2024-04-23T01:48:53.346665+0000","last_fullsized":"2024-04-23T01:48:53.346665+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"36'615","reported_seq":628,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:05.958251+0000","last_change":"2024-04-23T01:48:52.473323+0000","last_active":"2024-04-23T01:50:05.958251+0000","last_peered":"2024-04-23T01:50:05.958251+0000","last_clean":"2024-04-23T01:50:05.958251+0000","last_became_active":"2024-04-23T01:48:52.466602+0000","last_became_peered":"2024-04-23T01:48:52.466602+0000","last_unstale":"2024-04-23T01:50:05.958251+0000","last_undegraded":"2024-04-23T01:50:05.958251+0000","last_fullsized":"2024-04-23T01:50:05.958251+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5046272,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":616,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.439757+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:48:53.439757+0000","last_peered":"2024-04-23T01:48:53.439757+0000","last_clean":"2024-04-23T01:48:53.439757+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:48:53.439757+0000","last_undegraded":"2024-04-23T01:48:53.439757+0000","last_fullsized":"2024-04-23T01:48:53.439757+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"36'559","reported_seq":572,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:05.721781+0000","last_change":"2024-04-23T01:48:52.464660+0000","last_active":"2024-04-23T01:50:05.721781+0000","last_peered":"2024-04-23T01:50:05.721781+0000","last_clean":"2024-04-23T01:50:05.721781+0000","last_became_active":"2024-04-23T01:48:52.455507+0000","last_became_peered":"2024-04-23T01:48:52.455507+0000","last_unstale":"2024-04-23T01:50:05.721781+0000","last_undegraded":"2024-04-23T01:50:05.721781+0000","last_fullsized":"2024-04-23T01:50:05.721781+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4587520,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":560,"num_write_kb":4480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.430351+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:48:53.430351+0000","last_peered":"2024-04-23T01:48:53.430351+0000","last_clean":"2024-04-23T01:48:53.430351+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:48:53.430351+0000","last_undegraded":"2024-04-23T01:48:53.430351+0000","last_fullsized":"2024-04-23T01:48:53.430351+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":87670784,"num_objects":1340,"num_object_clones":0,"num_object_copies":4020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10702,"num_write_kb":85616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131510272,"data_stored":263020544,"data_compressed":13577931,"data_compressed_allocated":131510272,"data_compressed_original":263020544,"omap_allocated":0,"internal_metadata":0},"log_size":10686,"ondisk_log_size":10686,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":109552,"kb_used_data":29992,"kb_used_omap":0,"kb_used_meta":79552,"kb_avail":94262288,"statfs":{"total":96636764160,"available":96524582912,"internally_reserved":0,"allocated":30711808,"data_stored":61381618,"data_compressed":3162717,"data_compressed_allocated":30384128,"data_compressed_original":61259776,"omap_allocated":0,"internal_metadata":81461248},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":82676,"kb_used_data":37804,"kb_used_omap":0,"kb_used_meta":44864,"kb_avail":94289164,"statfs":{"total":96636764160,"available":96552103936,"internally_reserved":0,"allocated":38711296,"data_stored":77380594,"data_compressed":3988706,"data_compressed_allocated":38383616,"data_compressed_original":77258752,"omap_allocated":0,"internal_metadata":45940736},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":75688,"kb_used_data":26656,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94296152,"statfs":{"total":96636764160,"available":96559259648,"internally_reserved":0,"allocated":27295744,"data_stored":54082002,"data_compressed":2786441,"data_compressed_allocated":26988544,"data_compressed_original":53977088,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":83328,"kb_used_data":34296,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94288512,"statfs":{"total":96636764160,"available":96551436288,"internally_reserved":0,"allocated":35119104,"data_stored":70196210,"data_compressed":3617831,"data_compressed_allocated":34791424,"data_compressed_original":70074368,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":35041280,"data_stored":70082560,"data_compressed":3617911,"data_compressed_allocated":35041280,"data_compressed_original":70082560,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":27291648,"data_stored":54583296,"data_compressed":2817743,"data_compressed_allocated":27291648,"data_compressed_original":54583296,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":38719488,"data_stored":77438976,"data_compressed":3997669,"data_compressed_allocated":38719488,"data_compressed_original":77438976,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":30457856,"data_stored":60915712,"data_compressed":3144608,"data_compressed_allocated":30457856,"data_compressed_original":60915712,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:50:09.409 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 76 16 11088 11072 1.13806 1 0.174357 0.109704 2024-04-23T01:50:10.134 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:50:10.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:10.272+0000 7f8bb682d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:10.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:10.280+0000 7f8bb682d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:10.409 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 77 16 11208 11192 1.13545 0.9375 0.276091 0.109968 2024-04-23T01:50:11.409 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 78 16 11343 11327 1.13442 1.05469 0.145124 0.110094 2024-04-23T01:50:12.409 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 79 16 11479 11463 1.13351 1.0625 0.200523 0.110158 2024-04-23T01:50:12.464 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:50:12.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:12.579+0000 7fee9106d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:12.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:12.586+0000 7fee9106d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:13.409 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:50:13.410679+0000 min lat: 0.014864 max lat: 0.355605 avg lat: 0.110348 2024-04-23T01:50:13.410 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:50:13.410 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 80 16 11600 11584 1.13115 0.945312 0.182118 0.110348 2024-04-23T01:50:13.840 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:50:13.840 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:50:13.841 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:50:13.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:13.959+0000 7fec8659d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:13.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:13.968+0000 7fec8659d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:14.199 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:50:14.199 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:50:14.214 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":183,"stamp":"2024-04-23T01:50:13.607382+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":93594144,"num_objects":1422,"num_object_clones":0,"num_object_copies":4266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":11490,"num_write_kb":92718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11434,"ondisk_log_size":11434,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":372864,"kb_used_data":136480,"kb_used_omap":0,"kb_used_meta":236352,"kb_avail":377114496,"statfs":{"total":386547056640,"available":386165243904,"internally_reserved":0,"allocated":139755520,"data_stored":278875560,"data_compressed":14373351,"data_compressed_allocated":138465280,"data_compressed_original":278405120,"omap_allocated":0,"internal_metadata":242024448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16424960,"num_objects":250,"num_object_clones":0,"num_object_copies":750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2005,"num_write_kb":16040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002231"},"pg_stats":[{"pgid":"5.f","version":"36'687","reported_seq":700,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:11.329657+0000","last_change":"2024-04-23T01:48:52.530638+0000","last_active":"2024-04-23T01:50:11.329657+0000","last_peered":"2024-04-23T01:50:11.329657+0000","last_clean":"2024-04-23T01:50:11.329657+0000","last_became_active":"2024-04-23T01:48:52.520759+0000","last_became_peered":"2024-04-23T01:48:52.520759+0000","last_unstale":"2024-04-23T01:50:11.329657+0000","last_undegraded":"2024-04-23T01:50:11.329657+0000","last_fullsized":"2024-04-23T01:50:11.329657+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5636096,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":688,"num_write_kb":5504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"36'855","reported_seq":868,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:11.232996+0000","last_change":"2024-04-23T01:48:52.532479+0000","last_active":"2024-04-23T01:50:11.232996+0000","last_peered":"2024-04-23T01:50:11.232996+0000","last_clean":"2024-04-23T01:50:11.232996+0000","last_became_active":"2024-04-23T01:48:52.525595+0000","last_became_peered":"2024-04-23T01:48:52.525595+0000","last_unstale":"2024-04-23T01:50:11.232996+0000","last_undegraded":"2024-04-23T01:50:11.232996+0000","last_fullsized":"2024-04-23T01:50:11.232996+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7012352,"num_objects":107,"num_object_clones":0,"num_object_copies":321,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":856,"num_write_kb":6848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"36'783","reported_seq":796,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:10.450140+0000","last_change":"2024-04-23T01:48:52.571199+0000","last_active":"2024-04-23T01:50:10.450140+0000","last_peered":"2024-04-23T01:50:10.450140+0000","last_clean":"2024-04-23T01:50:10.450140+0000","last_became_active":"2024-04-23T01:48:52.566277+0000","last_became_peered":"2024-04-23T01:48:52.566277+0000","last_unstale":"2024-04-23T01:50:10.450140+0000","last_undegraded":"2024-04-23T01:50:10.450140+0000","last_fullsized":"2024-04-23T01:50:10.450140+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6422528,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"36'727","reported_seq":740,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:11.173195+0000","last_change":"2024-04-23T01:48:52.497141+0000","last_active":"2024-04-23T01:50:11.173195+0000","last_peered":"2024-04-23T01:50:11.173195+0000","last_clean":"2024-04-23T01:50:11.173195+0000","last_became_active":"2024-04-23T01:48:52.489329+0000","last_became_peered":"2024-04-23T01:48:52.489329+0000","last_unstale":"2024-04-23T01:50:11.173195+0000","last_undegraded":"2024-04-23T01:50:11.173195+0000","last_fullsized":"2024-04-23T01:50:11.173195+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":728,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"36'647","reported_seq":660,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:11.025591+0000","last_change":"2024-04-23T01:48:52.534975+0000","last_active":"2024-04-23T01:50:11.025591+0000","last_peered":"2024-04-23T01:50:11.025591+0000","last_clean":"2024-04-23T01:50:11.025591+0000","last_became_active":"2024-04-23T01:48:52.530664+0000","last_became_peered":"2024-04-23T01:48:52.530664+0000","last_unstale":"2024-04-23T01:50:11.025591+0000","last_undegraded":"2024-04-23T01:50:11.025591+0000","last_fullsized":"2024-04-23T01:50:11.025591+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308416,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":648,"num_write_kb":5184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"36'639","reported_seq":652,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:10.884431+0000","last_change":"2024-04-23T01:48:52.451606+0000","last_active":"2024-04-23T01:50:10.884431+0000","last_peered":"2024-04-23T01:50:10.884431+0000","last_clean":"2024-04-23T01:50:10.884431+0000","last_became_active":"2024-04-23T01:48:52.439020+0000","last_became_peered":"2024-04-23T01:48:52.439020+0000","last_unstale":"2024-04-23T01:50:10.884431+0000","last_undegraded":"2024-04-23T01:50:10.884431+0000","last_fullsized":"2024-04-23T01:50:10.884431+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":640,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"36'639","reported_seq":652,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:11.479422+0000","last_change":"2024-04-23T01:48:52.481437+0000","last_active":"2024-04-23T01:50:11.479422+0000","last_peered":"2024-04-23T01:50:11.479422+0000","last_clean":"2024-04-23T01:50:11.479422+0000","last_became_active":"2024-04-23T01:48:52.473748+0000","last_became_peered":"2024-04-23T01:48:52.473748+0000","last_unstale":"2024-04-23T01:50:11.479422+0000","last_undegraded":"2024-04-23T01:50:11.479422+0000","last_fullsized":"2024-04-23T01:50:11.479422+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":640,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"36'783","reported_seq":796,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:11.098179+0000","last_change":"2024-04-23T01:48:52.568521+0000","last_active":"2024-04-23T01:50:11.098179+0000","last_peered":"2024-04-23T01:50:11.098179+0000","last_clean":"2024-04-23T01:50:11.098179+0000","last_became_active":"2024-04-23T01:48:52.562008+0000","last_became_peered":"2024-04-23T01:48:52.562008+0000","last_unstale":"2024-04-23T01:50:11.098179+0000","last_undegraded":"2024-04-23T01:50:11.098179+0000","last_fullsized":"2024-04-23T01:50:11.098179+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6422528,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"36'695","reported_seq":708,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:10.354261+0000","last_change":"2024-04-23T01:48:52.491272+0000","last_active":"2024-04-23T01:50:10.354261+0000","last_peered":"2024-04-23T01:50:10.354261+0000","last_clean":"2024-04-23T01:50:10.354261+0000","last_became_active":"2024-04-23T01:48:52.482878+0000","last_became_peered":"2024-04-23T01:48:52.482878+0000","last_unstale":"2024-04-23T01:50:10.354261+0000","last_undegraded":"2024-04-23T01:50:10.354261+0000","last_fullsized":"2024-04-23T01:50:10.354261+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":696,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":203,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.336526+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:48:53.336526+0000","last_peered":"2024-04-23T01:48:53.336526+0000","last_clean":"2024-04-23T01:48:53.336526+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:48:53.336526+0000","last_undegraded":"2024-04-23T01:48:53.336526+0000","last_fullsized":"2024-04-23T01:48:53.336526+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.441413+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:48:53.441413+0000","last_peered":"2024-04-23T01:48:53.441413+0000","last_clean":"2024-04-23T01:48:53.441413+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:48:53.441413+0000","last_undegraded":"2024-04-23T01:48:53.441413+0000","last_fullsized":"2024-04-23T01:48:53.441413+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"36'783","reported_seq":796,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:11.420719+0000","last_change":"2024-04-23T01:48:52.550172+0000","last_active":"2024-04-23T01:50:11.420719+0000","last_peered":"2024-04-23T01:50:11.420719+0000","last_clean":"2024-04-23T01:50:11.420719+0000","last_became_active":"2024-04-23T01:48:52.541104+0000","last_became_peered":"2024-04-23T01:48:52.541104+0000","last_unstale":"2024-04-23T01:50:11.420719+0000","last_undegraded":"2024-04-23T01:50:11.420719+0000","last_fullsized":"2024-04-23T01:50:11.420719+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6422528,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.571998+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:48:53.571998+0000","last_peered":"2024-04-23T01:48:53.571998+0000","last_clean":"2024-04-23T01:48:53.571998+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:48:53.571998+0000","last_undegraded":"2024-04-23T01:48:53.571998+0000","last_fullsized":"2024-04-23T01:48:53.571998+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"36'663","reported_seq":676,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:10.967976+0000","last_change":"2024-04-23T01:48:52.526429+0000","last_active":"2024-04-23T01:50:10.967976+0000","last_peered":"2024-04-23T01:50:10.967976+0000","last_clean":"2024-04-23T01:50:10.967976+0000","last_became_active":"2024-04-23T01:48:52.514040+0000","last_became_peered":"2024-04-23T01:48:52.514040+0000","last_unstale":"2024-04-23T01:50:10.967976+0000","last_undegraded":"2024-04-23T01:50:10.967976+0000","last_fullsized":"2024-04-23T01:50:10.967976+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5439488,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":664,"num_write_kb":5312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.432658+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:48:53.432658+0000","last_peered":"2024-04-23T01:48:53.432658+0000","last_clean":"2024-04-23T01:48:53.432658+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:48:53.432658+0000","last_undegraded":"2024-04-23T01:48:53.432658+0000","last_fullsized":"2024-04-23T01:48:53.432658+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"36'871","reported_seq":884,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:11.098811+0000","last_change":"2024-04-23T01:48:52.469625+0000","last_active":"2024-04-23T01:50:11.098811+0000","last_peered":"2024-04-23T01:50:11.098811+0000","last_clean":"2024-04-23T01:50:11.098811+0000","last_became_active":"2024-04-23T01:48:52.456212+0000","last_became_peered":"2024-04-23T01:48:52.456212+0000","last_unstale":"2024-04-23T01:50:11.098811+0000","last_undegraded":"2024-04-23T01:50:11.098811+0000","last_fullsized":"2024-04-23T01:50:11.098811+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7143424,"num_objects":109,"num_object_clones":0,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":872,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":68,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.605161+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:48:53.605161+0000","last_peered":"2024-04-23T01:48:53.605161+0000","last_clean":"2024-04-23T01:48:53.605161+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:48:53.605161+0000","last_undegraded":"2024-04-23T01:48:53.605161+0000","last_fullsized":"2024-04-23T01:48:53.605161+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"36'644","reported_seq":657,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:11.582375+0000","last_change":"2024-04-23T01:48:52.505215+0000","last_active":"2024-04-23T01:50:11.582375+0000","last_peered":"2024-04-23T01:50:11.582375+0000","last_clean":"2024-04-23T01:50:11.582375+0000","last_became_active":"2024-04-23T01:48:52.497519+0000","last_became_peered":"2024-04-23T01:48:52.497519+0000","last_unstale":"2024-04-23T01:50:11.582375+0000","last_undegraded":"2024-04-23T01:50:11.582375+0000","last_fullsized":"2024-04-23T01:50:11.582375+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5283840,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":645,"num_write_kb":5160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.438431+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:48:53.438431+0000","last_peered":"2024-04-23T01:48:53.438431+0000","last_clean":"2024-04-23T01:48:53.438431+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:48:53.438431+0000","last_undegraded":"2024-04-23T01:48:53.438431+0000","last_fullsized":"2024-04-23T01:48:53.438431+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"36'695","reported_seq":708,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:09.955238+0000","last_change":"2024-04-23T01:48:52.501817+0000","last_active":"2024-04-23T01:50:09.955238+0000","last_peered":"2024-04-23T01:50:09.955238+0000","last_clean":"2024-04-23T01:50:09.955238+0000","last_became_active":"2024-04-23T01:48:52.496130+0000","last_became_peered":"2024-04-23T01:48:52.496130+0000","last_unstale":"2024-04-23T01:50:09.955238+0000","last_undegraded":"2024-04-23T01:50:09.955238+0000","last_fullsized":"2024-04-23T01:50:09.955238+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":696,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.346665+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:48:53.346665+0000","last_peered":"2024-04-23T01:48:53.346665+0000","last_clean":"2024-04-23T01:48:53.346665+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:48:53.346665+0000","last_undegraded":"2024-04-23T01:48:53.346665+0000","last_fullsized":"2024-04-23T01:48:53.346665+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"36'639","reported_seq":652,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:11.372373+0000","last_change":"2024-04-23T01:48:52.473323+0000","last_active":"2024-04-23T01:50:11.372373+0000","last_peered":"2024-04-23T01:50:11.372373+0000","last_clean":"2024-04-23T01:50:11.372373+0000","last_became_active":"2024-04-23T01:48:52.466602+0000","last_became_peered":"2024-04-23T01:48:52.466602+0000","last_unstale":"2024-04-23T01:50:11.372373+0000","last_undegraded":"2024-04-23T01:50:11.372373+0000","last_fullsized":"2024-04-23T01:50:11.372373+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":640,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.439757+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:48:53.439757+0000","last_peered":"2024-04-23T01:48:53.439757+0000","last_clean":"2024-04-23T01:48:53.439757+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:48:53.439757+0000","last_undegraded":"2024-04-23T01:48:53.439757+0000","last_fullsized":"2024-04-23T01:48:53.439757+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"36'587","reported_seq":600,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:11.590362+0000","last_change":"2024-04-23T01:48:52.464660+0000","last_active":"2024-04-23T01:50:11.590362+0000","last_peered":"2024-04-23T01:50:11.590362+0000","last_clean":"2024-04-23T01:50:11.590362+0000","last_became_active":"2024-04-23T01:48:52.455507+0000","last_became_peered":"2024-04-23T01:48:52.455507+0000","last_unstale":"2024-04-23T01:50:11.590362+0000","last_undegraded":"2024-04-23T01:50:11.590362+0000","last_fullsized":"2024-04-23T01:50:11.590362+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":587,"log_dups_size":0,"ondisk_log_size":587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4816896,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":588,"num_write_kb":4704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.430351+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:48:53.430351+0000","last_peered":"2024-04-23T01:48:53.430351+0000","last_clean":"2024-04-23T01:48:53.430351+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:48:53.430351+0000","last_undegraded":"2024-04-23T01:48:53.430351+0000","last_fullsized":"2024-04-23T01:48:53.430351+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":93003776,"num_objects":1420,"num_object_clones":0,"num_object_copies":4260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11353,"num_write_kb":90824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139509760,"data_stored":279019520,"data_compressed":14404047,"data_compressed_allocated":139509760,"data_compressed_original":279019520,"omap_allocated":0,"internal_metadata":0},"log_size":11337,"ondisk_log_size":11337,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":94468,"kb_used_data":31548,"kb_used_omap":0,"kb_used_meta":62912,"kb_avail":94277372,"statfs":{"total":96636764160,"available":96540028928,"internally_reserved":0,"allocated":32305152,"data_stored":64568306,"data_compressed":3327263,"data_compressed_allocated":31977472,"data_compressed_original":64446464,"omap_allocated":0,"internal_metadata":64421888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":89052,"kb_used_data":40020,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":94282788,"statfs":{"total":96636764160,"available":96545574912,"internally_reserved":0,"allocated":40980480,"data_stored":81918962,"data_compressed":4223047,"data_compressed_allocated":40652800,"data_compressed_original":81797120,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":81540,"kb_used_data":28348,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94290300,"statfs":{"total":96636764160,"available":96553267200,"internally_reserved":0,"allocated":29028352,"data_stored":57547218,"data_compressed":2965370,"data_compressed_allocated":28721152,"data_compressed_original":57442304,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":107804,"kb_used_data":36564,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94264036,"statfs":{"total":96636764160,"available":96526372864,"internally_reserved":0,"allocated":37441536,"data_stored":74841074,"data_compressed":3857671,"data_compressed_allocated":37113856,"data_compressed_original":74719232,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":37306368,"data_stored":74612736,"data_compressed":3851829,"data_compressed_allocated":37306368,"data_compressed_original":74612736,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":29007872,"data_stored":58015744,"data_compressed":2994980,"data_compressed_allocated":29007872,"data_compressed_original":58015744,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40996864,"data_stored":81993728,"data_compressed":4232856,"data_compressed_allocated":40996864,"data_compressed_original":81993728,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32198656,"data_stored":64397312,"data_compressed":3324382,"data_compressed_allocated":32198656,"data_compressed_original":64397312,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:50:14.410 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 81 16 11744 11728 1.13107 1.125 0.0681512 0.110431 2024-04-23T01:50:14.784 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:50:14.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:14.905+0000 7fdc5a15d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:14.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:14.912+0000 7fdc5a15d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:15.410 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 82 16 11873 11857 1.12957 1.00781 0.116048 0.110552 2024-04-23T01:50:16.410 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 83 16 12010 11994 1.12886 1.07031 0.114941 0.110643 2024-04-23T01:50:17.098 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:50:17.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:17.240+0000 7f9642c06640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:17.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:17.248+0000 7f9642c06640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:17.410 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 84 16 12150 12134 1.12844 1.09375 0.145051 0.110648 2024-04-23T01:50:18.410 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 85 16 12293 12277 1.1283 1.11719 0.100856 0.110671 2024-04-23T01:50:19.220 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:50:19.220 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:50:19.220 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:50:19.220 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:50:19.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:19.348+0000 7f96fcb5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:19.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:19.359+0000 7f96fcb5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:19.401 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:50:19.401 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:50:19.401 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:50:19.401 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:50:19.401 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:50:19.401 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:50:19.402 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:50:19.402 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:50:19.402 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:50:19.402 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:50:19.402 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:50:19.402 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:50:19.404 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:50:19.405 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:50:19.405 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:50:19.410 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 86 16 12433 12417 1.1279 1.09375 0.130777 0.110744 2024-04-23T01:50:19.476 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:50:19.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:19.539+0000 7f0732057640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:19.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:19.552+0000 7f0732057640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:19.590 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:50:19.590 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:50:19.590 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:50:19.590 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:50:19.590 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:50:19.590 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:50:19.590 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:50:19.590 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:50:19.590 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:50:19.591 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:50:19.591 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:50:19.591 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:50:19.594 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:50:19.594 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:50:19.594 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:50:19.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:19.620+0000 7f878846d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:19.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:19.628+0000 7f878846d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:19.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:19.722+0000 7f3bb986d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:19.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:19.733+0000 7f3bb986d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:19.831 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:50:19.831 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:50:19.831 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:50:19.831 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:50:19.831 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:50:19.831 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:50:19.831 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:50:19.831 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:50:19.831 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:50:19.832 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:50:19.832 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:50:19.832 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:50:19.833 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:50:19.833 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:50:19.834 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:50:20.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:19.996+0000 7f0a5fe6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:20.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:20.007+0000 7f0a5fe6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:20.025 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:50:20.026 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:50:20.026 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:50:20.026 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:50:20.026 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:50:20.026 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:50:20.026 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:50:20.026 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:50:20.026 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:50:20.026 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:50:20.026 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:50:20.026 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:50:20.029 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:50:20.029 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:50:20.410 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 87 16 12555 12539 1.12589 0.953125 0.0501779 0.11091 2024-04-23T01:50:21.410 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 88 16 12689 12673 1.12499 1.04688 0.095777 0.111002 2024-04-23T01:50:21.844 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:50:21.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:21.989+0000 7f6413c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:21.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:21.995+0000 7f6413c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:22.410 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 89 16 12823 12807 1.12411 1.04688 0.170734 0.111111 2024-04-23T01:50:23.410 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 90 15 12952 12937 1.12291 1.01562 0.191885 0.111232 2024-04-23T01:50:24.199 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:50:24.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:24.306+0000 7f1740657640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:24.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:24.311+0000 7f1740657640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:24.410 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Total time run: 90.1503 2024-04-23T01:50:24.411 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Total writes made: 12952 2024-04-23T01:50:24.411 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Write size: 8192 2024-04-23T01:50:24.411 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Object size: 65536 2024-04-23T01:50:24.411 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Bandwidth (MB/sec): 1.12243 2024-04-23T01:50:24.411 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Stddev Bandwidth: 0.0979888 2024-04-23T01:50:24.411 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Max bandwidth (MB/sec): 1.42188 2024-04-23T01:50:24.411 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Min bandwidth (MB/sec): 0.9375 2024-04-23T01:50:24.411 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Average IOPS: 143 2024-04-23T01:50:24.411 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Stddev IOPS: 12.5426 2024-04-23T01:50:24.411 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Max IOPS: 182 2024-04-23T01:50:24.411 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Min IOPS: 120 2024-04-23T01:50:24.411 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Average Latency(s): 0.111262 2024-04-23T01:50:24.411 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Stddev Latency(s): 0.0550671 2024-04-23T01:50:24.411 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Max latency(s): 0.355605 2024-04-23T01:50:24.412 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Min latency(s): 0.014864 2024-04-23T01:50:24.434 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Cleaning up (deleting benchmark objects) 2024-04-23T01:50:25.030 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:50:25.030 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:50:25.031 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:50:25.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:25.169+0000 7fac308ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:25.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:25.179+0000 7fac308ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:25.464 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:50:25.465 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:50:25.483 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-04-23T01:50:23.609330+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":104686112,"num_objects":1592,"num_object_clones":0,"num_object_copies":4776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":12844,"num_write_kb":103550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12788,"ondisk_log_size":12788,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":441932,"kb_used_data":152808,"kb_used_omap":0,"kb_used_meta":289088,"kb_avail":377045428,"statfs":{"total":386547056640,"available":386094518272,"internally_reserved":0,"allocated":156475392,"data_stored":312315304,"data_compressed":16100037,"data_compressed_allocated":155185152,"data_compressed_original":311844864,"omap_allocated":0,"internal_metadata":296026112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16424960,"num_objects":250,"num_object_clones":0,"num_object_copies":750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2005,"num_write_kb":16040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002285"},"pg_stats":[{"pgid":"5.f","version":"36'727","reported_seq":740,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:21.503865+0000","last_change":"2024-04-23T01:48:52.530638+0000","last_active":"2024-04-23T01:50:21.503865+0000","last_peered":"2024-04-23T01:50:21.503865+0000","last_clean":"2024-04-23T01:50:21.503865+0000","last_became_active":"2024-04-23T01:48:52.520759+0000","last_became_peered":"2024-04-23T01:48:52.520759+0000","last_unstale":"2024-04-23T01:50:21.503865+0000","last_undegraded":"2024-04-23T01:50:21.503865+0000","last_fullsized":"2024-04-23T01:50:21.503865+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":728,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"36'963","reported_seq":976,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:21.587576+0000","last_change":"2024-04-23T01:48:52.532479+0000","last_active":"2024-04-23T01:50:21.587576+0000","last_peered":"2024-04-23T01:50:21.587576+0000","last_clean":"2024-04-23T01:50:21.587576+0000","last_became_active":"2024-04-23T01:48:52.525595+0000","last_became_peered":"2024-04-23T01:48:52.525595+0000","last_unstale":"2024-04-23T01:50:21.587576+0000","last_undegraded":"2024-04-23T01:50:21.587576+0000","last_fullsized":"2024-04-23T01:50:21.587576+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":963,"log_dups_size":0,"ondisk_log_size":963,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7897088,"num_objects":121,"num_object_clones":0,"num_object_copies":363,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":964,"num_write_kb":7712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"36'897","reported_seq":910,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:21.587388+0000","last_change":"2024-04-23T01:48:52.571199+0000","last_active":"2024-04-23T01:50:21.587388+0000","last_peered":"2024-04-23T01:50:21.587388+0000","last_clean":"2024-04-23T01:50:21.587388+0000","last_became_active":"2024-04-23T01:48:52.566277+0000","last_became_peered":"2024-04-23T01:48:52.566277+0000","last_unstale":"2024-04-23T01:50:21.587388+0000","last_undegraded":"2024-04-23T01:50:21.587388+0000","last_fullsized":"2024-04-23T01:50:21.587388+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":897,"log_dups_size":0,"ondisk_log_size":897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7356416,"num_objects":113,"num_object_clones":0,"num_object_copies":339,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":898,"num_write_kb":7184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"36'783","reported_seq":796,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:20.921675+0000","last_change":"2024-04-23T01:48:52.497141+0000","last_active":"2024-04-23T01:50:20.921675+0000","last_peered":"2024-04-23T01:50:20.921675+0000","last_clean":"2024-04-23T01:50:20.921675+0000","last_became_active":"2024-04-23T01:48:52.489329+0000","last_became_peered":"2024-04-23T01:48:52.489329+0000","last_unstale":"2024-04-23T01:50:20.921675+0000","last_undegraded":"2024-04-23T01:50:20.921675+0000","last_fullsized":"2024-04-23T01:50:20.921675+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6422528,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"36'751","reported_seq":764,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:20.727605+0000","last_change":"2024-04-23T01:48:52.534975+0000","last_active":"2024-04-23T01:50:20.727605+0000","last_peered":"2024-04-23T01:50:20.727605+0000","last_clean":"2024-04-23T01:50:20.727605+0000","last_became_active":"2024-04-23T01:48:52.530664+0000","last_became_peered":"2024-04-23T01:48:52.530664+0000","last_unstale":"2024-04-23T01:50:20.727605+0000","last_undegraded":"2024-04-23T01:50:20.727605+0000","last_fullsized":"2024-04-23T01:50:20.727605+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"36'724","reported_seq":737,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:21.539315+0000","last_change":"2024-04-23T01:48:52.451606+0000","last_active":"2024-04-23T01:50:21.539315+0000","last_peered":"2024-04-23T01:50:21.539315+0000","last_clean":"2024-04-23T01:50:21.539315+0000","last_became_active":"2024-04-23T01:48:52.439020+0000","last_became_peered":"2024-04-23T01:48:52.439020+0000","last_unstale":"2024-04-23T01:50:21.539315+0000","last_undegraded":"2024-04-23T01:50:21.539315+0000","last_fullsized":"2024-04-23T01:50:21.539315+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5939200,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":725,"num_write_kb":5800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"36'711","reported_seq":724,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:21.498388+0000","last_change":"2024-04-23T01:48:52.481437+0000","last_active":"2024-04-23T01:50:21.498388+0000","last_peered":"2024-04-23T01:50:21.498388+0000","last_clean":"2024-04-23T01:50:21.498388+0000","last_became_active":"2024-04-23T01:48:52.473748+0000","last_became_peered":"2024-04-23T01:48:52.473748+0000","last_unstale":"2024-04-23T01:50:21.498388+0000","last_undegraded":"2024-04-23T01:50:21.498388+0000","last_fullsized":"2024-04-23T01:50:21.498388+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5832704,"num_objects":89,"num_object_clones":0,"num_object_copies":267,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":712,"num_write_kb":5696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"36'895","reported_seq":908,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:20.151025+0000","last_change":"2024-04-23T01:48:52.568521+0000","last_active":"2024-04-23T01:50:20.151025+0000","last_peered":"2024-04-23T01:50:20.151025+0000","last_clean":"2024-04-23T01:50:20.151025+0000","last_became_active":"2024-04-23T01:48:52.562008+0000","last_became_peered":"2024-04-23T01:48:52.562008+0000","last_unstale":"2024-04-23T01:50:20.151025+0000","last_undegraded":"2024-04-23T01:50:20.151025+0000","last_fullsized":"2024-04-23T01:50:20.151025+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340032,"num_objects":112,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":896,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"36'727","reported_seq":740,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:19.331862+0000","last_change":"2024-04-23T01:48:52.491272+0000","last_active":"2024-04-23T01:50:19.331862+0000","last_peered":"2024-04-23T01:50:19.331862+0000","last_clean":"2024-04-23T01:50:19.331862+0000","last_became_active":"2024-04-23T01:48:52.482878+0000","last_became_peered":"2024-04-23T01:48:52.482878+0000","last_unstale":"2024-04-23T01:50:19.331862+0000","last_undegraded":"2024-04-23T01:50:19.331862+0000","last_fullsized":"2024-04-23T01:50:19.331862+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":728,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":203,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.336526+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:48:53.336526+0000","last_peered":"2024-04-23T01:48:53.336526+0000","last_clean":"2024-04-23T01:48:53.336526+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:48:53.336526+0000","last_undegraded":"2024-04-23T01:48:53.336526+0000","last_fullsized":"2024-04-23T01:48:53.336526+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.441413+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:48:53.441413+0000","last_peered":"2024-04-23T01:48:53.441413+0000","last_clean":"2024-04-23T01:48:53.441413+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:48:53.441413+0000","last_undegraded":"2024-04-23T01:48:53.441413+0000","last_fullsized":"2024-04-23T01:48:53.441413+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"36'887","reported_seq":900,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:21.331544+0000","last_change":"2024-04-23T01:48:52.550172+0000","last_active":"2024-04-23T01:50:21.331544+0000","last_peered":"2024-04-23T01:50:21.331544+0000","last_clean":"2024-04-23T01:50:21.331544+0000","last_became_active":"2024-04-23T01:48:52.541104+0000","last_became_peered":"2024-04-23T01:48:52.541104+0000","last_unstale":"2024-04-23T01:50:21.331544+0000","last_undegraded":"2024-04-23T01:50:21.331544+0000","last_fullsized":"2024-04-23T01:50:21.331544+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":887,"log_dups_size":0,"ondisk_log_size":887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7274496,"num_objects":111,"num_object_clones":0,"num_object_copies":333,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":888,"num_write_kb":7104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.571998+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:48:53.571998+0000","last_peered":"2024-04-23T01:48:53.571998+0000","last_clean":"2024-04-23T01:48:53.571998+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:48:53.571998+0000","last_undegraded":"2024-04-23T01:48:53.571998+0000","last_fullsized":"2024-04-23T01:48:53.571998+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"36'751","reported_seq":764,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:21.128680+0000","last_change":"2024-04-23T01:48:52.526429+0000","last_active":"2024-04-23T01:50:21.128680+0000","last_peered":"2024-04-23T01:50:21.128680+0000","last_clean":"2024-04-23T01:50:21.128680+0000","last_became_active":"2024-04-23T01:48:52.514040+0000","last_became_peered":"2024-04-23T01:48:52.514040+0000","last_unstale":"2024-04-23T01:50:21.128680+0000","last_undegraded":"2024-04-23T01:50:21.128680+0000","last_fullsized":"2024-04-23T01:50:21.128680+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.432658+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:48:53.432658+0000","last_peered":"2024-04-23T01:48:53.432658+0000","last_clean":"2024-04-23T01:48:53.432658+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:48:53.432658+0000","last_undegraded":"2024-04-23T01:48:53.432658+0000","last_fullsized":"2024-04-23T01:48:53.432658+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"36'983","reported_seq":996,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:21.404272+0000","last_change":"2024-04-23T01:48:52.469625+0000","last_active":"2024-04-23T01:50:21.404272+0000","last_peered":"2024-04-23T01:50:21.404272+0000","last_clean":"2024-04-23T01:50:21.404272+0000","last_became_active":"2024-04-23T01:48:52.456212+0000","last_became_peered":"2024-04-23T01:48:52.456212+0000","last_unstale":"2024-04-23T01:50:21.404272+0000","last_undegraded":"2024-04-23T01:50:21.404272+0000","last_fullsized":"2024-04-23T01:50:21.404272+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":983,"log_dups_size":0,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8060928,"num_objects":123,"num_object_clones":0,"num_object_copies":369,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":984,"num_write_kb":7872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":68,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.605161+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:48:53.605161+0000","last_peered":"2024-04-23T01:48:53.605161+0000","last_clean":"2024-04-23T01:48:53.605161+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:48:53.605161+0000","last_undegraded":"2024-04-23T01:48:53.605161+0000","last_fullsized":"2024-04-23T01:48:53.605161+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"36'743","reported_seq":756,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:21.030100+0000","last_change":"2024-04-23T01:48:52.505215+0000","last_active":"2024-04-23T01:50:21.030100+0000","last_peered":"2024-04-23T01:50:21.030100+0000","last_clean":"2024-04-23T01:50:21.030100+0000","last_became_active":"2024-04-23T01:48:52.497519+0000","last_became_peered":"2024-04-23T01:48:52.497519+0000","last_unstale":"2024-04-23T01:50:21.030100+0000","last_undegraded":"2024-04-23T01:50:21.030100+0000","last_fullsized":"2024-04-23T01:50:21.030100+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6094848,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":744,"num_write_kb":5952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.438431+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:48:53.438431+0000","last_peered":"2024-04-23T01:48:53.438431+0000","last_clean":"2024-04-23T01:48:53.438431+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:48:53.438431+0000","last_undegraded":"2024-04-23T01:48:53.438431+0000","last_fullsized":"2024-04-23T01:48:53.438431+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"36'743","reported_seq":756,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:21.075560+0000","last_change":"2024-04-23T01:48:52.501817+0000","last_active":"2024-04-23T01:50:21.075560+0000","last_peered":"2024-04-23T01:50:21.075560+0000","last_clean":"2024-04-23T01:50:21.075560+0000","last_became_active":"2024-04-23T01:48:52.496130+0000","last_became_peered":"2024-04-23T01:48:52.496130+0000","last_unstale":"2024-04-23T01:50:21.075560+0000","last_undegraded":"2024-04-23T01:50:21.075560+0000","last_fullsized":"2024-04-23T01:50:21.075560+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6094848,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":744,"num_write_kb":5952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.346665+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:48:53.346665+0000","last_peered":"2024-04-23T01:48:53.346665+0000","last_clean":"2024-04-23T01:48:53.346665+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:48:53.346665+0000","last_undegraded":"2024-04-23T01:48:53.346665+0000","last_fullsized":"2024-04-23T01:48:53.346665+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"36'727","reported_seq":740,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:19.561752+0000","last_change":"2024-04-23T01:48:52.473323+0000","last_active":"2024-04-23T01:50:19.561752+0000","last_peered":"2024-04-23T01:50:19.561752+0000","last_clean":"2024-04-23T01:50:19.561752+0000","last_became_active":"2024-04-23T01:48:52.466602+0000","last_became_peered":"2024-04-23T01:48:52.466602+0000","last_unstale":"2024-04-23T01:50:19.561752+0000","last_undegraded":"2024-04-23T01:50:19.561752+0000","last_fullsized":"2024-04-23T01:50:19.561752+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":728,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.439757+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:48:53.439757+0000","last_peered":"2024-04-23T01:48:53.439757+0000","last_clean":"2024-04-23T01:48:53.439757+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:48:53.439757+0000","last_undegraded":"2024-04-23T01:48:53.439757+0000","last_fullsized":"2024-04-23T01:48:53.439757+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"36'679","reported_seq":692,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:21.571455+0000","last_change":"2024-04-23T01:48:52.464660+0000","last_active":"2024-04-23T01:50:21.571455+0000","last_peered":"2024-04-23T01:50:21.571455+0000","last_clean":"2024-04-23T01:50:21.571455+0000","last_became_active":"2024-04-23T01:48:52.455507+0000","last_became_peered":"2024-04-23T01:48:52.455507+0000","last_unstale":"2024-04-23T01:50:21.571455+0000","last_undegraded":"2024-04-23T01:50:21.571455+0000","last_fullsized":"2024-04-23T01:50:21.571455+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5570560,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":680,"num_write_kb":5440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.430351+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:48:53.430351+0000","last_peered":"2024-04-23T01:48:53.430351+0000","last_clean":"2024-04-23T01:48:53.430351+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:48:53.430351+0000","last_undegraded":"2024-04-23T01:48:53.430351+0000","last_fullsized":"2024-04-23T01:48:53.430351+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":104095744,"num_objects":1590,"num_object_clones":0,"num_object_copies":4770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12707,"num_write_kb":101656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":156135424,"data_stored":312270848,"data_compressed":16121004,"data_compressed_allocated":156135424,"data_compressed_original":312270848,"omap_allocated":0,"internal_metadata":0},"log_size":12691,"ondisk_log_size":12691,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705978,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":102368,"kb_used_data":35288,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94269472,"statfs":{"total":96636764160,"available":96531939328,"internally_reserved":0,"allocated":36134912,"data_stored":72227826,"data_compressed":3722768,"data_compressed_allocated":35807232,"data_compressed_original":72105984,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705978,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":116108,"kb_used_data":44868,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94255732,"statfs":{"total":96636764160,"available":96517869568,"internally_reserved":0,"allocated":45944832,"data_stored":91847666,"data_compressed":4735723,"data_compressed_allocated":45617152,"data_compressed_original":91725824,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705978,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":107180,"kb_used_data":31780,"kb_used_omap":0,"kb_used_meta":75392,"kb_avail":94264660,"statfs":{"total":96636764160,"available":96527011840,"internally_reserved":0,"allocated":32542720,"data_stored":64575954,"data_compressed":3328304,"data_compressed_allocated":32235520,"data_compressed_original":64471040,"omap_allocated":0,"internal_metadata":77201408},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705978,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":116276,"kb_used_data":40872,"kb_used_omap":0,"kb_used_meta":75392,"kb_avail":94255564,"statfs":{"total":96636764160,"available":96517697536,"internally_reserved":0,"allocated":41852928,"data_stored":83663858,"data_compressed":4313242,"data_compressed_allocated":41525248,"data_compressed_original":83542016,"omap_allocated":0,"internal_metadata":77201408},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":41762816,"data_stored":83525632,"data_compressed":4312053,"data_compressed_allocated":41762816,"data_compressed_original":83525632,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32407552,"data_stored":64815104,"data_compressed":3346070,"data_compressed_allocated":32407552,"data_compressed_original":64815104,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45916160,"data_stored":91832320,"data_compressed":4740879,"data_compressed_allocated":45916160,"data_compressed_original":91832320,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":36048896,"data_stored":72097792,"data_compressed":3722002,"data_compressed_allocated":36048896,"data_compressed_original":72097792,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:50:26.408 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:50:26.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:26.565+0000 7f21d16ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:26.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:26.572+0000 7f21d16ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:29.041 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:50:29.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:29.175+0000 7f7c3266d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:29.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:29.182+0000 7f7c3140b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:30.485 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:50:30.485 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:50:30.485 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:50:30.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:30.613+0000 7fcc1eb5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:30.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:30.622+0000 7fcc1eb5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:30.941 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:50:30.941 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:50:30.959 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-04-23T01:50:29.610568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":82051640,"num_objects":1246,"num_object_clones":0,"num_object_copies":3738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":13466,"num_write_kb":105511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13410,"ondisk_log_size":13410,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":425040,"kb_used_data":131728,"kb_used_omap":0,"kb_used_meta":293248,"kb_avail":377062320,"statfs":{"total":386547056640,"available":386111815680,"internally_reserved":0,"allocated":134889472,"data_stored":269118960,"data_compressed":13872888,"data_compressed_allocated":133586944,"data_compressed_original":268648448,"omap_allocated":0,"internal_metadata":300285952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17080296,"num_objects":-261,"num_object_clones":0,"num_object_copies":-783,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1300,"num_write_kb":7385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002460"},"pg_stats":[{"pgid":"5.f","version":"36'751","reported_seq":764,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:26.435228+0000","last_change":"2024-04-23T01:48:52.530638+0000","last_active":"2024-04-23T01:50:26.435228+0000","last_peered":"2024-04-23T01:50:26.435228+0000","last_clean":"2024-04-23T01:50:26.435228+0000","last_became_active":"2024-04-23T01:48:52.520759+0000","last_became_peered":"2024-04-23T01:48:52.520759+0000","last_unstale":"2024-04-23T01:50:26.435228+0000","last_undegraded":"2024-04-23T01:50:26.435228+0000","last_fullsized":"2024-04-23T01:50:26.435228+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4980736,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":5888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"36'986","reported_seq":999,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:26.267185+0000","last_change":"2024-04-23T01:48:52.532479+0000","last_active":"2024-04-23T01:50:26.267185+0000","last_peered":"2024-04-23T01:50:26.267185+0000","last_clean":"2024-04-23T01:50:26.267185+0000","last_became_active":"2024-04-23T01:48:52.525595+0000","last_became_peered":"2024-04-23T01:48:52.525595+0000","last_unstale":"2024-04-23T01:50:26.267185+0000","last_undegraded":"2024-04-23T01:50:26.267185+0000","last_fullsized":"2024-04-23T01:50:26.267185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":986,"log_dups_size":0,"ondisk_log_size":986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6684672,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":987,"num_write_kb":7744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"36'935","reported_seq":949,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:26.477866+0000","last_change":"2024-04-23T01:48:52.571199+0000","last_active":"2024-04-23T01:50:26.477866+0000","last_peered":"2024-04-23T01:50:26.477866+0000","last_clean":"2024-04-23T01:50:26.477866+0000","last_became_active":"2024-04-23T01:48:52.566277+0000","last_became_peered":"2024-04-23T01:48:52.566277+0000","last_unstale":"2024-04-23T01:50:26.477866+0000","last_undegraded":"2024-04-23T01:50:26.477866+0000","last_fullsized":"2024-04-23T01:50:26.477866+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5373976,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":936,"num_write_kb":7233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"36'820","reported_seq":833,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:26.503909+0000","last_change":"2024-04-23T01:48:52.497141+0000","last_active":"2024-04-23T01:50:26.503909+0000","last_peered":"2024-04-23T01:50:26.503909+0000","last_clean":"2024-04-23T01:50:26.503909+0000","last_became_active":"2024-04-23T01:48:52.489329+0000","last_became_peered":"2024-04-23T01:48:52.489329+0000","last_unstale":"2024-04-23T01:50:26.503909+0000","last_undegraded":"2024-04-23T01:50:26.503909+0000","last_fullsized":"2024-04-23T01:50:26.503909+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":820,"log_dups_size":0,"ondisk_log_size":820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":821,"num_write_kb":6400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"36'790","reported_seq":803,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:26.231481+0000","last_change":"2024-04-23T01:48:52.534975+0000","last_active":"2024-04-23T01:50:26.231481+0000","last_peered":"2024-04-23T01:50:26.231481+0000","last_clean":"2024-04-23T01:50:26.231481+0000","last_became_active":"2024-04-23T01:48:52.530664+0000","last_became_peered":"2024-04-23T01:48:52.530664+0000","last_unstale":"2024-04-23T01:50:26.231481+0000","last_undegraded":"2024-04-23T01:50:26.231481+0000","last_fullsized":"2024-04-23T01:50:26.231481+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4784128,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":791,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"36'781","reported_seq":794,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:26.360219+0000","last_change":"2024-04-23T01:48:52.451606+0000","last_active":"2024-04-23T01:50:26.360219+0000","last_peered":"2024-04-23T01:50:26.360219+0000","last_clean":"2024-04-23T01:50:26.360219+0000","last_became_active":"2024-04-23T01:48:52.439020+0000","last_became_peered":"2024-04-23T01:48:52.439020+0000","last_unstale":"2024-04-23T01:50:26.360219+0000","last_undegraded":"2024-04-23T01:50:26.360219+0000","last_fullsized":"2024-04-23T01:50:26.360219+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4784128,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":782,"num_write_kb":6080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"36'746","reported_seq":759,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:26.526700+0000","last_change":"2024-04-23T01:48:52.481437+0000","last_active":"2024-04-23T01:50:26.526700+0000","last_peered":"2024-04-23T01:50:26.526700+0000","last_clean":"2024-04-23T01:50:26.526700+0000","last_became_active":"2024-04-23T01:48:52.473748+0000","last_became_peered":"2024-04-23T01:48:52.473748+0000","last_unstale":"2024-04-23T01:50:26.526700+0000","last_undegraded":"2024-04-23T01:50:26.526700+0000","last_fullsized":"2024-04-23T01:50:26.526700+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":746,"log_dups_size":0,"ondisk_log_size":746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718592,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":747,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"36'930","reported_seq":943,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:26.500981+0000","last_change":"2024-04-23T01:48:52.568521+0000","last_active":"2024-04-23T01:50:26.500981+0000","last_peered":"2024-04-23T01:50:26.500981+0000","last_clean":"2024-04-23T01:50:26.500981+0000","last_became_active":"2024-04-23T01:48:52.562008+0000","last_became_peered":"2024-04-23T01:48:52.562008+0000","last_unstale":"2024-04-23T01:50:26.500981+0000","last_undegraded":"2024-04-23T01:50:26.500981+0000","last_fullsized":"2024-04-23T01:50:26.500981+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":930,"log_dups_size":0,"ondisk_log_size":930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6225920,"num_objects":95,"num_object_clones":0,"num_object_copies":285,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":931,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"36'767","reported_seq":780,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:26.418872+0000","last_change":"2024-04-23T01:48:52.491272+0000","last_active":"2024-04-23T01:50:26.418872+0000","last_peered":"2024-04-23T01:50:26.418872+0000","last_clean":"2024-04-23T01:50:26.418872+0000","last_became_active":"2024-04-23T01:48:52.482878+0000","last_became_peered":"2024-04-23T01:48:52.482878+0000","last_unstale":"2024-04-23T01:50:26.418872+0000","last_undegraded":"2024-04-23T01:50:26.418872+0000","last_fullsized":"2024-04-23T01:50:26.418872+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4521984,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":768,"num_write_kb":5952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":203,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.336526+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:48:53.336526+0000","last_peered":"2024-04-23T01:48:53.336526+0000","last_clean":"2024-04-23T01:48:53.336526+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:48:53.336526+0000","last_undegraded":"2024-04-23T01:48:53.336526+0000","last_fullsized":"2024-04-23T01:48:53.336526+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.441413+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:48:53.441413+0000","last_peered":"2024-04-23T01:48:53.441413+0000","last_clean":"2024-04-23T01:48:53.441413+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:48:53.441413+0000","last_undegraded":"2024-04-23T01:48:53.441413+0000","last_fullsized":"2024-04-23T01:48:53.441413+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"36'922","reported_seq":935,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:26.550128+0000","last_change":"2024-04-23T01:48:52.550172+0000","last_active":"2024-04-23T01:50:26.550128+0000","last_peered":"2024-04-23T01:50:26.550128+0000","last_clean":"2024-04-23T01:50:26.550128+0000","last_became_active":"2024-04-23T01:48:52.541104+0000","last_became_peered":"2024-04-23T01:48:52.541104+0000","last_unstale":"2024-04-23T01:50:26.550128+0000","last_undegraded":"2024-04-23T01:50:26.550128+0000","last_fullsized":"2024-04-23T01:50:26.550128+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5570560,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":923,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.571998+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:48:53.571998+0000","last_peered":"2024-04-23T01:48:53.571998+0000","last_clean":"2024-04-23T01:48:53.571998+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:48:53.571998+0000","last_undegraded":"2024-04-23T01:48:53.571998+0000","last_fullsized":"2024-04-23T01:48:53.571998+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"36'782","reported_seq":795,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:26.581696+0000","last_change":"2024-04-23T01:48:52.526429+0000","last_active":"2024-04-23T01:50:26.581696+0000","last_peered":"2024-04-23T01:50:26.581696+0000","last_clean":"2024-04-23T01:50:26.581696+0000","last_became_active":"2024-04-23T01:48:52.514040+0000","last_became_peered":"2024-04-23T01:48:52.514040+0000","last_unstale":"2024-04-23T01:50:26.581696+0000","last_undegraded":"2024-04-23T01:50:26.581696+0000","last_fullsized":"2024-04-23T01:50:26.581696+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718592,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":783,"num_write_kb":6080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.432658+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:48:53.432658+0000","last_peered":"2024-04-23T01:48:53.432658+0000","last_clean":"2024-04-23T01:48:53.432658+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:48:53.432658+0000","last_undegraded":"2024-04-23T01:48:53.432658+0000","last_fullsized":"2024-04-23T01:48:53.432658+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"36'1027","reported_seq":1040,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:26.538508+0000","last_change":"2024-04-23T01:48:52.469625+0000","last_active":"2024-04-23T01:50:26.538508+0000","last_peered":"2024-04-23T01:50:26.538508+0000","last_clean":"2024-04-23T01:50:26.538508+0000","last_became_active":"2024-04-23T01:48:52.456212+0000","last_became_peered":"2024-04-23T01:48:52.456212+0000","last_unstale":"2024-04-23T01:50:26.538508+0000","last_undegraded":"2024-04-23T01:50:26.538508+0000","last_fullsized":"2024-04-23T01:50:26.538508+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":1027,"log_dups_size":0,"ondisk_log_size":1027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767168,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1028,"num_write_kb":7936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":68,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.605161+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:48:53.605161+0000","last_peered":"2024-04-23T01:48:53.605161+0000","last_clean":"2024-04-23T01:48:53.605161+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:48:53.605161+0000","last_undegraded":"2024-04-23T01:48:53.605161+0000","last_fullsized":"2024-04-23T01:48:53.605161+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"36'783","reported_seq":796,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:26.463087+0000","last_change":"2024-04-23T01:48:52.505215+0000","last_active":"2024-04-23T01:50:26.463087+0000","last_peered":"2024-04-23T01:50:26.463087+0000","last_clean":"2024-04-23T01:50:26.463087+0000","last_became_active":"2024-04-23T01:48:52.497519+0000","last_became_peered":"2024-04-23T01:48:52.497519+0000","last_unstale":"2024-04-23T01:50:26.463087+0000","last_undegraded":"2024-04-23T01:50:26.463087+0000","last_fullsized":"2024-04-23T01:50:26.463087+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":6080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.438431+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:48:53.438431+0000","last_peered":"2024-04-23T01:48:53.438431+0000","last_clean":"2024-04-23T01:48:53.438431+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:48:53.438431+0000","last_undegraded":"2024-04-23T01:48:53.438431+0000","last_fullsized":"2024-04-23T01:48:53.438431+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"36'795","reported_seq":808,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:26.559072+0000","last_change":"2024-04-23T01:48:52.501817+0000","last_active":"2024-04-23T01:50:26.559072+0000","last_peered":"2024-04-23T01:50:26.559072+0000","last_clean":"2024-04-23T01:50:26.559072+0000","last_became_active":"2024-04-23T01:48:52.496130+0000","last_became_peered":"2024-04-23T01:48:52.496130+0000","last_unstale":"2024-04-23T01:50:26.559072+0000","last_undegraded":"2024-04-23T01:50:26.559072+0000","last_fullsized":"2024-04-23T01:50:26.559072+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":795,"log_dups_size":0,"ondisk_log_size":795,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":796,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.346665+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:48:53.346665+0000","last_peered":"2024-04-23T01:48:53.346665+0000","last_clean":"2024-04-23T01:48:53.346665+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:48:53.346665+0000","last_undegraded":"2024-04-23T01:48:53.346665+0000","last_fullsized":"2024-04-23T01:48:53.346665+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"36'765","reported_seq":778,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:26.448857+0000","last_change":"2024-04-23T01:48:52.473323+0000","last_active":"2024-04-23T01:50:26.448857+0000","last_peered":"2024-04-23T01:50:26.448857+0000","last_clean":"2024-04-23T01:50:26.448857+0000","last_became_active":"2024-04-23T01:48:52.466602+0000","last_became_peered":"2024-04-23T01:48:52.466602+0000","last_unstale":"2024-04-23T01:50:26.448857+0000","last_undegraded":"2024-04-23T01:50:26.448857+0000","last_fullsized":"2024-04-23T01:50:26.448857+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":765,"log_dups_size":0,"ondisk_log_size":765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":766,"num_write_kb":5952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.439757+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:48:53.439757+0000","last_peered":"2024-04-23T01:48:53.439757+0000","last_clean":"2024-04-23T01:48:53.439757+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:48:53.439757+0000","last_undegraded":"2024-04-23T01:48:53.439757+0000","last_fullsized":"2024-04-23T01:48:53.439757+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"36'733","reported_seq":746,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:50:26.432803+0000","last_change":"2024-04-23T01:48:52.464660+0000","last_active":"2024-04-23T01:50:26.432803+0000","last_peered":"2024-04-23T01:50:26.432803+0000","last_clean":"2024-04-23T01:50:26.432803+0000","last_became_active":"2024-04-23T01:48:52.455507+0000","last_became_peered":"2024-04-23T01:48:52.455507+0000","last_unstale":"2024-04-23T01:50:26.432803+0000","last_undegraded":"2024-04-23T01:50:26.432803+0000","last_fullsized":"2024-04-23T01:50:26.432803+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:48:51.310292+0000","last_clean_scrub_stamp":"2024-04-23T01:48:51.310292+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4390912,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":734,"num_write_kb":5696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.430351+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:48:53.430351+0000","last_peered":"2024-04-23T01:48:53.430351+0000","last_clean":"2024-04-23T01:48:53.430351+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:48:53.430351+0000","last_undegraded":"2024-04-23T01:48:53.430351+0000","last_fullsized":"2024-04-23T01:48:53.430351+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":81461272,"num_objects":1244,"num_object_clones":0,"num_object_copies":3732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13329,"num_write_kb":103617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122171392,"data_stored":244318280,"data_compressed":12615549,"data_compressed_allocated":122159104,"data_compressed_original":244318208,"omap_allocated":0,"internal_metadata":0},"log_size":13313,"ondisk_log_size":13313,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705983,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":97396,"kb_used_data":30316,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94274444,"statfs":{"total":96636764160,"available":96537030656,"internally_reserved":0,"allocated":31043584,"data_stored":62037002,"data_compressed":3197376,"data_compressed_allocated":30711808,"data_compressed_original":61915136,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705983,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":114420,"kb_used_data":39020,"kb_used_omap":0,"kb_used_meta":75392,"kb_avail":94257420,"statfs":{"total":96636764160,"available":96519598080,"internally_reserved":0,"allocated":39956480,"data_stored":79862794,"data_compressed":4117824,"data_compressed_allocated":39624704,"data_compressed_original":79740928,"omap_allocated":0,"internal_metadata":77201408},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705983,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":102740,"kb_used_data":27340,"kb_used_omap":0,"kb_used_meta":75392,"kb_avail":94269100,"statfs":{"total":96636764160,"available":96531558400,"internally_reserved":0,"allocated":27996160,"data_stored":55482834,"data_compressed":2859480,"data_compressed_allocated":27688960,"data_compressed_original":55377920,"omap_allocated":0,"internal_metadata":77201408},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705983,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":110484,"kb_used_data":35052,"kb_used_omap":0,"kb_used_meta":75392,"kb_avail":94261356,"statfs":{"total":96636764160,"available":96523628544,"internally_reserved":0,"allocated":35893248,"data_stored":71736330,"data_compressed":3698208,"data_compressed_allocated":35561472,"data_compressed_original":71614464,"omap_allocated":0,"internal_metadata":77201408},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32935936,"data_stored":65863704,"data_compressed":3400919,"data_compressed_allocated":32931840,"data_compressed_original":65863680,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":25100288,"data_stored":50200576,"data_compressed":2592144,"data_compressed_allocated":25100288,"data_compressed_original":50200576,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36081664,"data_stored":72155160,"data_compressed":3725783,"data_compressed_allocated":36077568,"data_compressed_original":72155136,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28053504,"data_stored":56098840,"data_compressed":2896703,"data_compressed_allocated":28049408,"data_compressed_original":56098816,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:50:31.507 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:50:31.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:31.660+0000 7f07f9dff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:31.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:31.668+0000 7f07f9dff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:33.947 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:50:34.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:34.078+0000 7f48c262d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:34.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:34.085+0000 7f48c262d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:34.435 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Removed 1619 objects 2024-04-23T01:50:34.436 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Clean up completed and total clean up time :10.0015 2024-04-23T01:50:35.960 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:50:35.961 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:50:35.961 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:50:35.961 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:50:36.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:36.083+0000 7f35956ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:36.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:36.092+0000 7f35956ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:36.109 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:50:36.109 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:50:36.109 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:50:36.109 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:50:36.109 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:50:36.109 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:50:36.109 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:50:36.109 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:50:36.109 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:50:36.109 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:50:36.109 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:50:36.110 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:50:36.112 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:50:36.112 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:50:36.112 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:50:36.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:36.217+0000 7f30714ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:36.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:36.226+0000 7f30714ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:36.242 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:50:36.242 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:50:36.242 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:50:36.242 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:50:36.242 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:50:36.242 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:50:36.242 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:50:36.243 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:50:36.243 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:50:36.243 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:50:36.243 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:50:36.243 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:50:36.245 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:50:36.245 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:50:36.246 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:50:36.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:36.347+0000 7fcc21a17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:36.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:36.356+0000 7fcc21a17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:36.374 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:50:36.374 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:50:36.374 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:50:36.374 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:50:36.374 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:50:36.374 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:50:36.374 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:50:36.375 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:50:36.375 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:50:36.375 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:50:36.375 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:50:36.375 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:50:36.377 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:50:36.377 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:50:36.377 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:50:36.461 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:50:36.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:36.485+0000 7eff3186d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:36.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:36.494+0000 7eff3186d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:36.513 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:50:36.513 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:50:36.513 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:50:36.513 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:50:36.513 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:50:36.513 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:50:36.513 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:50:36.513 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:50:36.513 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:50:36.514 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:50:36.514 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:50:36.514 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:50:36.516 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:50:36.516 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:50:36.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:36.573+0000 7f353239d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:36.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:36.578+0000 7f353239d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:37.066 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_2 2024-04-23T01:50:37.066 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_2 unique_pool_2 --yes-i-really-really-mean-it 2024-04-23T01:50:37.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:37.170+0000 7f4c9859d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:37.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:37.182+0000 7f4c9859d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:38.256 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'unique_pool_2' removed 2024-04-23T01:50:38.272 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-04-23T01:50:38.273 INFO:tasks.radosbench:Beginning radosbench... 2024-04-23T01:50:38.273 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_3 2024-04-23T01:50:38.273 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_3 16 2024-04-23T01:50:38.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:38.382+0000 7f263a82d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:38.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:38.392+0000 7f263a82d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:38.678 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:50:38.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:38.784+0000 7f67b08ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:38.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:38.790+0000 7f67b08ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:39.263 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'unique_pool_3' created 2024-04-23T01:50:39.280 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_3 rados --yes-i-really-mean-it || true 2024-04-23T01:50:39.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:39.416+0000 7fceca8ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:39.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:39.429+0000 7fceca8ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:40.266 INFO:teuthology.orchestra.run.smithi090.stderr:enabled application 'rados' on pool 'unique_pool_3' 2024-04-23T01:50:41.075 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:50:41.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:41.181+0000 7f5ecaa2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:41.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:41.186+0000 7f5ecaa2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:41.288 DEBUG:teuthology.orchestra.run.smithi090:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 8192 --object-size 65536 -p unique_pool_3 bench 90 write' 2024-04-23T01:50:41.290 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-04-23T01:50:41.291 INFO:teuthology.orchestra.run:waiting for 3000 2024-04-23T01:50:41.350 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:hints = 1 2024-04-23T01:50:41.350 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Maintaining 16 concurrent writes of 8192 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-04-23T01:50:41.350 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Object prefix: benchmark_data_smithi090_41641 2024-04-23T01:50:41.350 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:50:41.350 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 0 0 0 0 0 0 - 0 2024-04-23T01:50:41.517 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:50:41.517 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:50:41.518 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:50:41.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:41.647+0000 7feefee6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:41.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:41.659+0000 7feefee6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:41.972 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:50:41.973 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:50:41.991 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-04-23T01:50:41.613465+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":278020,"kb_used_data":1380,"kb_used_omap":0,"kb_used_meta":276608,"kb_avail":377209340,"statfs":{"total":386547056640,"available":386262364160,"internally_reserved":0,"allocated":1413120,"data_stored":2190760,"data_compressed":89859,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":283246592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-14573,"num_write_kb":-103617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001106"},"pg_stats":[{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:50:39.257496+0000","last_change":"2024-04-23T01:50:39.257496+0000","last_active":"2024-04-23T01:50:39.257496+0000","last_peered":"2024-04-23T01:50:39.257496+0000","last_clean":"2024-04-23T01:50:39.257496+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:50:39.257496+0000","last_undegraded":"2024-04-23T01:50:39.257496+0000","last_fullsized":"2024-04-23T01:50:39.257496+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:50:39.257496+0000","last_change":"2024-04-23T01:50:39.257496+0000","last_active":"2024-04-23T01:50:39.257496+0000","last_peered":"2024-04-23T01:50:39.257496+0000","last_clean":"2024-04-23T01:50:39.257496+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:50:39.257496+0000","last_undegraded":"2024-04-23T01:50:39.257496+0000","last_fullsized":"2024-04-23T01:50:39.257496+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:50:39.257496+0000","last_change":"2024-04-23T01:50:39.257496+0000","last_active":"2024-04-23T01:50:39.257496+0000","last_peered":"2024-04-23T01:50:39.257496+0000","last_clean":"2024-04-23T01:50:39.257496+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:50:39.257496+0000","last_undegraded":"2024-04-23T01:50:39.257496+0000","last_fullsized":"2024-04-23T01:50:39.257496+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:50:39.257496+0000","last_change":"2024-04-23T01:50:39.257496+0000","last_active":"2024-04-23T01:50:39.257496+0000","last_peered":"2024-04-23T01:50:39.257496+0000","last_clean":"2024-04-23T01:50:39.257496+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:50:39.257496+0000","last_undegraded":"2024-04-23T01:50:39.257496+0000","last_fullsized":"2024-04-23T01:50:39.257496+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:50:39.257496+0000","last_change":"2024-04-23T01:50:39.257496+0000","last_active":"2024-04-23T01:50:39.257496+0000","last_peered":"2024-04-23T01:50:39.257496+0000","last_clean":"2024-04-23T01:50:39.257496+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:50:39.257496+0000","last_undegraded":"2024-04-23T01:50:39.257496+0000","last_fullsized":"2024-04-23T01:50:39.257496+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:50:39.257496+0000","last_change":"2024-04-23T01:50:39.257496+0000","last_active":"2024-04-23T01:50:39.257496+0000","last_peered":"2024-04-23T01:50:39.257496+0000","last_clean":"2024-04-23T01:50:39.257496+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:50:39.257496+0000","last_undegraded":"2024-04-23T01:50:39.257496+0000","last_fullsized":"2024-04-23T01:50:39.257496+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:50:39.257496+0000","last_change":"2024-04-23T01:50:39.257496+0000","last_active":"2024-04-23T01:50:39.257496+0000","last_peered":"2024-04-23T01:50:39.257496+0000","last_clean":"2024-04-23T01:50:39.257496+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:50:39.257496+0000","last_undegraded":"2024-04-23T01:50:39.257496+0000","last_fullsized":"2024-04-23T01:50:39.257496+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:50:39.257496+0000","last_change":"2024-04-23T01:50:39.257496+0000","last_active":"2024-04-23T01:50:39.257496+0000","last_peered":"2024-04-23T01:50:39.257496+0000","last_clean":"2024-04-23T01:50:39.257496+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:50:39.257496+0000","last_undegraded":"2024-04-23T01:50:39.257496+0000","last_fullsized":"2024-04-23T01:50:39.257496+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:50:39.257496+0000","last_change":"2024-04-23T01:50:39.257496+0000","last_active":"2024-04-23T01:50:39.257496+0000","last_peered":"2024-04-23T01:50:39.257496+0000","last_clean":"2024-04-23T01:50:39.257496+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:50:39.257496+0000","last_undegraded":"2024-04-23T01:50:39.257496+0000","last_fullsized":"2024-04-23T01:50:39.257496+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":203,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.336526+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:48:53.336526+0000","last_peered":"2024-04-23T01:48:53.336526+0000","last_clean":"2024-04-23T01:48:53.336526+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:48:53.336526+0000","last_undegraded":"2024-04-23T01:48:53.336526+0000","last_fullsized":"2024-04-23T01:48:53.336526+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.441413+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:48:53.441413+0000","last_peered":"2024-04-23T01:48:53.441413+0000","last_clean":"2024-04-23T01:48:53.441413+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:48:53.441413+0000","last_undegraded":"2024-04-23T01:48:53.441413+0000","last_fullsized":"2024-04-23T01:48:53.441413+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:50:39.257496+0000","last_change":"2024-04-23T01:50:39.257496+0000","last_active":"2024-04-23T01:50:39.257496+0000","last_peered":"2024-04-23T01:50:39.257496+0000","last_clean":"2024-04-23T01:50:39.257496+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:50:39.257496+0000","last_undegraded":"2024-04-23T01:50:39.257496+0000","last_fullsized":"2024-04-23T01:50:39.257496+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.571998+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:48:53.571998+0000","last_peered":"2024-04-23T01:48:53.571998+0000","last_clean":"2024-04-23T01:48:53.571998+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:48:53.571998+0000","last_undegraded":"2024-04-23T01:48:53.571998+0000","last_fullsized":"2024-04-23T01:48:53.571998+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:50:39.257496+0000","last_change":"2024-04-23T01:50:39.257496+0000","last_active":"2024-04-23T01:50:39.257496+0000","last_peered":"2024-04-23T01:50:39.257496+0000","last_clean":"2024-04-23T01:50:39.257496+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:50:39.257496+0000","last_undegraded":"2024-04-23T01:50:39.257496+0000","last_fullsized":"2024-04-23T01:50:39.257496+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.432658+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:48:53.432658+0000","last_peered":"2024-04-23T01:48:53.432658+0000","last_clean":"2024-04-23T01:48:53.432658+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:48:53.432658+0000","last_undegraded":"2024-04-23T01:48:53.432658+0000","last_fullsized":"2024-04-23T01:48:53.432658+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:50:39.257496+0000","last_change":"2024-04-23T01:50:39.257496+0000","last_active":"2024-04-23T01:50:39.257496+0000","last_peered":"2024-04-23T01:50:39.257496+0000","last_clean":"2024-04-23T01:50:39.257496+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:50:39.257496+0000","last_undegraded":"2024-04-23T01:50:39.257496+0000","last_fullsized":"2024-04-23T01:50:39.257496+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":68,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.605161+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:48:53.605161+0000","last_peered":"2024-04-23T01:48:53.605161+0000","last_clean":"2024-04-23T01:48:53.605161+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:48:53.605161+0000","last_undegraded":"2024-04-23T01:48:53.605161+0000","last_fullsized":"2024-04-23T01:48:53.605161+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:50:39.257496+0000","last_change":"2024-04-23T01:50:39.257496+0000","last_active":"2024-04-23T01:50:39.257496+0000","last_peered":"2024-04-23T01:50:39.257496+0000","last_clean":"2024-04-23T01:50:39.257496+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:50:39.257496+0000","last_undegraded":"2024-04-23T01:50:39.257496+0000","last_fullsized":"2024-04-23T01:50:39.257496+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.438431+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:48:53.438431+0000","last_peered":"2024-04-23T01:48:53.438431+0000","last_clean":"2024-04-23T01:48:53.438431+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:48:53.438431+0000","last_undegraded":"2024-04-23T01:48:53.438431+0000","last_fullsized":"2024-04-23T01:48:53.438431+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:50:39.257496+0000","last_change":"2024-04-23T01:50:39.257496+0000","last_active":"2024-04-23T01:50:39.257496+0000","last_peered":"2024-04-23T01:50:39.257496+0000","last_clean":"2024-04-23T01:50:39.257496+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:50:39.257496+0000","last_undegraded":"2024-04-23T01:50:39.257496+0000","last_fullsized":"2024-04-23T01:50:39.257496+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.346665+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:48:53.346665+0000","last_peered":"2024-04-23T01:48:53.346665+0000","last_clean":"2024-04-23T01:48:53.346665+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:48:53.346665+0000","last_undegraded":"2024-04-23T01:48:53.346665+0000","last_fullsized":"2024-04-23T01:48:53.346665+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:50:39.257496+0000","last_change":"2024-04-23T01:50:39.257496+0000","last_active":"2024-04-23T01:50:39.257496+0000","last_peered":"2024-04-23T01:50:39.257496+0000","last_clean":"2024-04-23T01:50:39.257496+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:50:39.257496+0000","last_undegraded":"2024-04-23T01:50:39.257496+0000","last_fullsized":"2024-04-23T01:50:39.257496+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.439757+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:48:53.439757+0000","last_peered":"2024-04-23T01:48:53.439757+0000","last_clean":"2024-04-23T01:48:53.439757+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:48:53.439757+0000","last_undegraded":"2024-04-23T01:48:53.439757+0000","last_fullsized":"2024-04-23T01:48:53.439757+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-23T01:50:39.257496+0000","last_change":"2024-04-23T01:50:39.257496+0000","last_active":"2024-04-23T01:50:39.257496+0000","last_peered":"2024-04-23T01:50:39.257496+0000","last_clean":"2024-04-23T01:50:39.257496+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:50:39.257496+0000","last_undegraded":"2024-04-23T01:50:39.257496+0000","last_fullsized":"2024-04-23T01:50:39.257496+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-23T01:48:53.430351+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:48:53.430351+0000","last_peered":"2024-04-23T01:48:53.430351+0000","last_clean":"2024-04-23T01:48:53.430351+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:48:53.430351+0000","last_undegraded":"2024-04-23T01:48:53.430351+0000","last_fullsized":"2024-04-23T01:48:53.430351+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705993,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":71600,"kb_used_data":360,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94300240,"statfs":{"total":96636764160,"available":96563445760,"internally_reserved":0,"allocated":368640,"data_stored":695282,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705993,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":63280,"kb_used_data":360,"kb_used_omap":0,"kb_used_meta":62912,"kb_avail":94308560,"statfs":{"total":96636764160,"available":96571965440,"internally_reserved":0,"allocated":368640,"data_stored":695282,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":64421888},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705993,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":79860,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":79552,"kb_avail":94291980,"statfs":{"total":96636764160,"available":96554987520,"internally_reserved":0,"allocated":307200,"data_stored":104914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":81461248},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705993,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":63280,"kb_used_data":360,"kb_used_omap":0,"kb_used_meta":62912,"kb_avail":94308560,"statfs":{"total":96636764160,"available":96571965440,"internally_reserved":0,"allocated":368640,"data_stored":695282,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":64421888},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:50:42.350 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 1 16 155 139 1.08588 1.08594 0.11466 0.109204 2024-04-23T01:50:43.283 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:50:43.350 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 2 16 315 299 1.1679 1.25 0.0482377 0.10292 2024-04-23T01:50:43.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:43.399+0000 7f7000a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:43.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:43.406+0000 7f7000a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:44.351 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 3 16 464 448 1.16659 1.16406 0.0825032 0.104595 2024-04-23T01:50:45.351 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 4 16 627 611 1.19328 1.27344 0.0474521 0.103423 2024-04-23T01:50:45.650 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:50:45.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:45.771+0000 7effdc99d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:45.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:45.777+0000 7effdc99d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:46.351 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 5 16 784 768 1.19991 1.22656 0.122242 0.102767 2024-04-23T01:50:46.993 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:50:46.993 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:50:46.993 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:50:46.993 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:50:47.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:47.113+0000 7fda7b39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:47.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:47.127+0000 7fda7b39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:47.151 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:50:47.151 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:50:47.151 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:50:47.152 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:50:47.152 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:50:47.152 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:50:47.152 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:50:47.152 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:50:47.152 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:50:47.152 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:50:47.152 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:50:47.152 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:50:47.155 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:50:47.155 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:50:47.155 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:50:47.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:47.286+0000 7f486f0ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:47.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:47.299+0000 7f486f0ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:47.326 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:50:47.327 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:50:47.327 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:50:47.327 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:50:47.327 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:50:47.327 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:50:47.327 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:50:47.327 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:50:47.327 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:50:47.327 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:50:47.327 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:50:47.328 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:50:47.329 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:50:47.330 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:50:47.330 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:50:47.351 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 6 16 931 915 1.19132 1.14844 0.125781 0.103955 2024-04-23T01:50:47.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:47.485+0000 7fb17dc64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:47.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:47.497+0000 7fb17dc64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:47.514 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:50:47.514 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:50:47.514 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:50:47.514 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:50:47.514 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:50:47.514 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:50:47.514 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:50:47.514 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:50:47.514 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:50:47.515 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:50:47.515 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:50:47.515 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:50:47.517 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:50:47.517 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:50:47.517 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:50:47.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:47.680+0000 7fc7154ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:47.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:47.690+0000 7fc7154ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:47.742 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:50:47.742 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:50:47.742 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:50:47.742 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:50:47.743 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:50:47.743 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:50:47.743 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:50:47.743 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:50:47.743 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:50:47.743 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:50:47.743 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:50:47.743 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:50:47.745 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:50:47.746 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:50:47.993 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:50:48.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:48.148+0000 7f4e73c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:48.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:48.155+0000 7f4e73c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:48.351 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 7 16 1066 1050 1.17179 1.05469 0.148761 0.105375 2024-04-23T01:50:49.351 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 8 16 1212 1196 1.16787 1.14062 0.0680868 0.106233 2024-04-23T01:50:50.334 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:50:50.351 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 9 16 1385 1369 1.18827 1.35156 0.0694102 0.104512 2024-04-23T01:50:50.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:50.473+0000 7f8e0202d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:50.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:50.481+0000 7f8e0202d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:51.351 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 10 16 1524 1508 1.17803 1.08594 0.0464251 0.105498 2024-04-23T01:50:52.351 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 11 16 1687 1671 1.18669 1.27344 0.0689684 0.104852 2024-04-23T01:50:52.648 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:50:52.747 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:50:52.747 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:50:52.747 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:50:52.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:52.777+0000 7f598259d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:52.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:52.784+0000 7f598259d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:52.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:52.881+0000 7f6fa46ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:52.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:52.894+0000 7f6fa46ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:53.128 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:50:53.128 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:50:53.143 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":206,"stamp":"2024-04-23T01:50:51.615334+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7217696,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":946,"num_write_kb":8366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":890,"ondisk_log_size":890,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":278504,"kb_used_data":10184,"kb_used_omap":0,"kb_used_meta":268288,"kb_avail":377208856,"statfs":{"total":386547056640,"available":386261868544,"internally_reserved":0,"allocated":10428416,"data_stored":19982420,"data_compressed":1003159,"data_compressed_allocated":8990720,"data_compressed_original":19456000,"omap_allocated":0,"internal_metadata":274726912},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6627328,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":809,"num_write_kb":6472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.344612"},"pg_stats":[{"pgid":"6.f","version":"40'47","reported_seq":60,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:45.996982+0000","last_change":"2024-04-23T01:50:40.440701+0000","last_active":"2024-04-23T01:50:45.996982+0000","last_peered":"2024-04-23T01:50:45.996982+0000","last_clean":"2024-04-23T01:50:45.996982+0000","last_became_active":"2024-04-23T01:50:40.433832+0000","last_became_peered":"2024-04-23T01:50:40.433832+0000","last_unstale":"2024-04-23T01:50:45.996982+0000","last_undegraded":"2024-04-23T01:50:45.996982+0000","last_fullsized":"2024-04-23T01:50:45.996982+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"40'47","reported_seq":60,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:46.070982+0000","last_change":"2024-04-23T01:50:40.477278+0000","last_active":"2024-04-23T01:50:46.070982+0000","last_peered":"2024-04-23T01:50:46.070982+0000","last_clean":"2024-04-23T01:50:46.070982+0000","last_became_active":"2024-04-23T01:50:40.471274+0000","last_became_peered":"2024-04-23T01:50:40.471274+0000","last_unstale":"2024-04-23T01:50:46.070982+0000","last_undegraded":"2024-04-23T01:50:46.070982+0000","last_fullsized":"2024-04-23T01:50:46.070982+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"40'71","reported_seq":84,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:46.181498+0000","last_change":"2024-04-23T01:50:40.446224+0000","last_active":"2024-04-23T01:50:46.181498+0000","last_peered":"2024-04-23T01:50:46.181498+0000","last_clean":"2024-04-23T01:50:46.181498+0000","last_became_active":"2024-04-23T01:50:40.437838+0000","last_became_peered":"2024-04-23T01:50:40.437838+0000","last_unstale":"2024-04-23T01:50:46.181498+0000","last_undegraded":"2024-04-23T01:50:46.181498+0000","last_fullsized":"2024-04-23T01:50:46.181498+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"40'23","reported_seq":36,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:43.133532+0000","last_change":"2024-04-23T01:50:40.517607+0000","last_active":"2024-04-23T01:50:43.133532+0000","last_peered":"2024-04-23T01:50:43.133532+0000","last_clean":"2024-04-23T01:50:43.133532+0000","last_became_active":"2024-04-23T01:50:40.511881+0000","last_became_peered":"2024-04-23T01:50:40.511881+0000","last_unstale":"2024-04-23T01:50:43.133532+0000","last_undegraded":"2024-04-23T01:50:43.133532+0000","last_fullsized":"2024-04-23T01:50:43.133532+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"40'61","reported_seq":74,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:46.603965+0000","last_change":"2024-04-23T01:50:40.432817+0000","last_active":"2024-04-23T01:50:46.603965+0000","last_peered":"2024-04-23T01:50:46.603965+0000","last_clean":"2024-04-23T01:50:46.603965+0000","last_became_active":"2024-04-23T01:50:40.421722+0000","last_became_peered":"2024-04-23T01:50:40.421722+0000","last_unstale":"2024-04-23T01:50:46.603965+0000","last_undegraded":"2024-04-23T01:50:46.603965+0000","last_fullsized":"2024-04-23T01:50:46.603965+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":507904,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62,"num_write_kb":496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"40'31","reported_seq":44,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:44.960771+0000","last_change":"2024-04-23T01:50:40.444510+0000","last_active":"2024-04-23T01:50:44.960771+0000","last_peered":"2024-04-23T01:50:44.960771+0000","last_clean":"2024-04-23T01:50:44.960771+0000","last_became_active":"2024-04-23T01:50:40.437088+0000","last_became_peered":"2024-04-23T01:50:40.437088+0000","last_unstale":"2024-04-23T01:50:44.960771+0000","last_undegraded":"2024-04-23T01:50:44.960771+0000","last_fullsized":"2024-04-23T01:50:44.960771+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"40'79","reported_seq":92,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:46.144275+0000","last_change":"2024-04-23T01:50:40.468123+0000","last_active":"2024-04-23T01:50:46.144275+0000","last_peered":"2024-04-23T01:50:46.144275+0000","last_clean":"2024-04-23T01:50:46.144275+0000","last_became_active":"2024-04-23T01:50:40.443838+0000","last_became_peered":"2024-04-23T01:50:40.443838+0000","last_unstale":"2024-04-23T01:50:46.144275+0000","last_undegraded":"2024-04-23T01:50:46.144275+0000","last_fullsized":"2024-04-23T01:50:46.144275+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"40'23","reported_seq":36,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:45.016368+0000","last_change":"2024-04-23T01:50:40.451743+0000","last_active":"2024-04-23T01:50:45.016368+0000","last_peered":"2024-04-23T01:50:45.016368+0000","last_clean":"2024-04-23T01:50:45.016368+0000","last_became_active":"2024-04-23T01:50:40.447092+0000","last_became_peered":"2024-04-23T01:50:40.447092+0000","last_unstale":"2024-04-23T01:50:45.016368+0000","last_undegraded":"2024-04-23T01:50:45.016368+0000","last_fullsized":"2024-04-23T01:50:45.016368+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"40'23","reported_seq":36,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:45.610115+0000","last_change":"2024-04-23T01:50:40.477557+0000","last_active":"2024-04-23T01:50:45.610115+0000","last_peered":"2024-04-23T01:50:45.610115+0000","last_clean":"2024-04-23T01:50:45.610115+0000","last_became_active":"2024-04-23T01:50:40.472479+0000","last_became_peered":"2024-04-23T01:50:40.472479+0000","last_unstale":"2024-04-23T01:50:45.610115+0000","last_undegraded":"2024-04-23T01:50:45.610115+0000","last_fullsized":"2024-04-23T01:50:45.610115+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":215,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:41.626121+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:50:41.626121+0000","last_peered":"2024-04-23T01:50:41.626121+0000","last_clean":"2024-04-23T01:50:41.626121+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:50:41.626121+0000","last_undegraded":"2024-04-23T01:50:41.626121+0000","last_fullsized":"2024-04-23T01:50:41.626121+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":82,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:41.615653+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:50:41.615653+0000","last_peered":"2024-04-23T01:50:41.615653+0000","last_clean":"2024-04-23T01:50:41.615653+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:50:41.615653+0000","last_undegraded":"2024-04-23T01:50:41.615653+0000","last_fullsized":"2024-04-23T01:50:41.615653+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"40'46","reported_seq":59,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:46.582863+0000","last_change":"2024-04-23T01:50:40.431365+0000","last_active":"2024-04-23T01:50:46.582863+0000","last_peered":"2024-04-23T01:50:46.582863+0000","last_clean":"2024-04-23T01:50:46.582863+0000","last_became_active":"2024-04-23T01:50:40.426237+0000","last_became_peered":"2024-04-23T01:50:40.426237+0000","last_unstale":"2024-04-23T01:50:46.582863+0000","last_undegraded":"2024-04-23T01:50:46.582863+0000","last_fullsized":"2024-04-23T01:50:46.582863+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":385024,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":80,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:41.588811+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:50:41.588811+0000","last_peered":"2024-04-23T01:50:41.588811+0000","last_clean":"2024-04-23T01:50:41.588811+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:50:41.588811+0000","last_undegraded":"2024-04-23T01:50:41.588811+0000","last_fullsized":"2024-04-23T01:50:41.588811+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"40'39","reported_seq":52,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:46.340773+0000","last_change":"2024-04-23T01:50:40.486856+0000","last_active":"2024-04-23T01:50:46.340773+0000","last_peered":"2024-04-23T01:50:46.340773+0000","last_clean":"2024-04-23T01:50:46.340773+0000","last_became_active":"2024-04-23T01:50:40.478453+0000","last_became_peered":"2024-04-23T01:50:40.478453+0000","last_unstale":"2024-04-23T01:50:46.340773+0000","last_undegraded":"2024-04-23T01:50:46.340773+0000","last_fullsized":"2024-04-23T01:50:46.340773+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":82,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:41.608295+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:50:41.608295+0000","last_peered":"2024-04-23T01:50:41.608295+0000","last_clean":"2024-04-23T01:50:41.608295+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:50:41.608295+0000","last_undegraded":"2024-04-23T01:50:41.608295+0000","last_fullsized":"2024-04-23T01:50:41.608295+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"40'71","reported_seq":84,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:46.395356+0000","last_change":"2024-04-23T01:50:40.462462+0000","last_active":"2024-04-23T01:50:46.395356+0000","last_peered":"2024-04-23T01:50:46.395356+0000","last_clean":"2024-04-23T01:50:46.395356+0000","last_became_active":"2024-04-23T01:50:40.439584+0000","last_became_peered":"2024-04-23T01:50:40.439584+0000","last_unstale":"2024-04-23T01:50:46.395356+0000","last_undegraded":"2024-04-23T01:50:46.395356+0000","last_fullsized":"2024-04-23T01:50:46.395356+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":80,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:41.597584+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:50:41.597584+0000","last_peered":"2024-04-23T01:50:41.597584+0000","last_clean":"2024-04-23T01:50:41.597584+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:50:41.597584+0000","last_undegraded":"2024-04-23T01:50:41.597584+0000","last_fullsized":"2024-04-23T01:50:41.597584+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"40'47","reported_seq":60,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:46.078477+0000","last_change":"2024-04-23T01:50:40.466468+0000","last_active":"2024-04-23T01:50:46.078477+0000","last_peered":"2024-04-23T01:50:46.078477+0000","last_clean":"2024-04-23T01:50:46.078477+0000","last_became_active":"2024-04-23T01:50:40.452408+0000","last_became_peered":"2024-04-23T01:50:40.452408+0000","last_unstale":"2024-04-23T01:50:46.078477+0000","last_undegraded":"2024-04-23T01:50:46.078477+0000","last_fullsized":"2024-04-23T01:50:46.078477+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":86,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:41.290654+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:50:41.290654+0000","last_peered":"2024-04-23T01:50:41.290654+0000","last_clean":"2024-04-23T01:50:41.290654+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:50:41.290654+0000","last_undegraded":"2024-04-23T01:50:41.290654+0000","last_fullsized":"2024-04-23T01:50:41.290654+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"40'35","reported_seq":48,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:46.577788+0000","last_change":"2024-04-23T01:50:40.469709+0000","last_active":"2024-04-23T01:50:46.577788+0000","last_peered":"2024-04-23T01:50:46.577788+0000","last_clean":"2024-04-23T01:50:46.577788+0000","last_became_active":"2024-04-23T01:50:40.462062+0000","last_became_peered":"2024-04-23T01:50:40.462062+0000","last_unstale":"2024-04-23T01:50:46.577788+0000","last_undegraded":"2024-04-23T01:50:46.577788+0000","last_fullsized":"2024-04-23T01:50:46.577788+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":294912,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":82,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:41.626809+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:50:41.626809+0000","last_peered":"2024-04-23T01:50:41.626809+0000","last_clean":"2024-04-23T01:50:41.626809+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:50:41.626809+0000","last_undegraded":"2024-04-23T01:50:41.626809+0000","last_fullsized":"2024-04-23T01:50:41.626809+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"40'63","reported_seq":76,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:45.202646+0000","last_change":"2024-04-23T01:50:40.464715+0000","last_active":"2024-04-23T01:50:45.202646+0000","last_peered":"2024-04-23T01:50:45.202646+0000","last_clean":"2024-04-23T01:50:45.202646+0000","last_became_active":"2024-04-23T01:50:40.457999+0000","last_became_peered":"2024-04-23T01:50:40.457999+0000","last_unstale":"2024-04-23T01:50:45.202646+0000","last_undegraded":"2024-04-23T01:50:45.202646+0000","last_fullsized":"2024-04-23T01:50:45.202646+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":86,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:41.311541+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:50:41.311541+0000","last_peered":"2024-04-23T01:50:41.311541+0000","last_clean":"2024-04-23T01:50:41.311541+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:50:41.311541+0000","last_undegraded":"2024-04-23T01:50:41.311541+0000","last_fullsized":"2024-04-23T01:50:41.311541+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"40'87","reported_seq":100,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:46.300204+0000","last_change":"2024-04-23T01:50:40.447151+0000","last_active":"2024-04-23T01:50:46.300204+0000","last_peered":"2024-04-23T01:50:46.300204+0000","last_clean":"2024-04-23T01:50:46.300204+0000","last_became_active":"2024-04-23T01:50:40.441283+0000","last_became_peered":"2024-04-23T01:50:40.441283+0000","last_unstale":"2024-04-23T01:50:46.300204+0000","last_undegraded":"2024-04-23T01:50:46.300204+0000","last_fullsized":"2024-04-23T01:50:46.300204+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":720896,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":82,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:41.624618+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:50:41.624618+0000","last_peered":"2024-04-23T01:50:41.624618+0000","last_clean":"2024-04-23T01:50:41.624618+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:50:41.624618+0000","last_undegraded":"2024-04-23T01:50:41.624618+0000","last_fullsized":"2024-04-23T01:50:41.624618+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":6627328,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":809,"num_write_kb":6472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":9945088,"data_stored":19890176,"data_compressed":1024286,"data_compressed_allocated":9945088,"data_compressed_original":19890176,"omap_allocated":0,"internal_metadata":0},"log_size":793,"ondisk_log_size":793,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706003,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77780,"kb_used_data":2380,"kb_used_omap":0,"kb_used_meta":75392,"kb_avail":94294060,"statfs":{"total":96636764160,"available":96557117440,"internally_reserved":0,"allocated":2437120,"data_stored":4772509,"data_compressed":239183,"data_compressed_allocated":2072576,"data_compressed_original":4636672,"omap_allocated":0,"internal_metadata":77201408},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706003,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65156,"kb_used_data":2236,"kb_used_omap":0,"kb_used_meta":62912,"kb_avail":94306684,"statfs":{"total":96636764160,"available":96570044416,"internally_reserved":0,"allocated":2289664,"data_stored":4477597,"data_compressed":223987,"data_compressed_allocated":1925120,"data_compressed_original":4341760,"omap_allocated":0,"internal_metadata":64421888},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706003,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65744,"kb_used_data":2824,"kb_used_omap":0,"kb_used_meta":62912,"kb_avail":94306096,"statfs":{"total":96636764160,"available":96569442304,"internally_reserved":0,"allocated":2891776,"data_stored":5214333,"data_compressed":262424,"data_compressed_allocated":2547712,"data_compressed_original":5095424,"omap_allocated":0,"internal_metadata":64421888},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706003,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":69824,"kb_used_data":2744,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94302016,"statfs":{"total":96636764160,"available":96565264384,"internally_reserved":0,"allocated":2809856,"data_stored":5517981,"data_compressed":277565,"data_compressed_allocated":2445312,"data_compressed_original":5382144,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2637824,"data_stored":5275648,"data_compressed":271666,"data_compressed_allocated":2637824,"data_compressed_original":5275648,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2863104,"data_stored":5726208,"data_compressed":294918,"data_compressed_allocated":2863104,"data_compressed_original":5726208,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2068480,"data_stored":4136960,"data_compressed":213024,"data_compressed_allocated":2068480,"data_compressed_original":4136960,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":4751360,"data_compressed":244678,"data_compressed_allocated":2375680,"data_compressed_original":4751360,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:50:53.351 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 12 16 1841 1825 1.18805 1.20312 0.24574 0.104618 2024-04-23T01:50:54.351 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 13 16 2010 1994 1.19822 1.32031 0.162772 0.103947 2024-04-23T01:50:54.973 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:50:55.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:55.125+0000 7f114c354640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:55.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:55.132+0000 7f114c354640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:55.352 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 14 16 2160 2144 1.19633 1.17188 0.0531021 0.10398 2024-04-23T01:50:56.352 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 15 16 2313 2297 1.19626 1.19531 0.0786015 0.103916 2024-04-23T01:50:57.335 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:50:57.351 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 16 16 2463 2447 1.19473 1.17188 0.105028 0.104233 2024-04-23T01:50:57.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:57.479+0000 7f923639d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:57.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:57.486+0000 7f923639d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:58.145 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:50:58.145 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:50:58.145 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:50:58.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:58.306+0000 7fa1d622d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:58.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:58.318+0000 7fa1d622d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:58.352 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 17 16 2601 2585 1.18786 1.07812 0.136725 0.104723 2024-04-23T01:50:58.555 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:50:58.556 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:50:58.571 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2024-04-23T01:50:57.616416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19751456,"num_objects":296,"num_object_clones":0,"num_object_copies":888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":2476,"num_write_kb":20606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2420,"ondisk_log_size":2420,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":301060,"kb_used_data":28572,"kb_used_omap":0,"kb_used_meta":272448,"kb_avail":377186300,"statfs":{"total":386547056640,"available":386238771200,"internally_reserved":0,"allocated":29257728,"data_stored":57641044,"data_compressed":2946855,"data_compressed_allocated":27820032,"data_compressed_original":57114624,"omap_allocated":0,"internal_metadata":278986752},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18964480,"num_objects":290,"num_object_clones":0,"num_object_copies":870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2315,"num_write_kb":18520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002158"},"pg_stats":[{"pgid":"6.f","version":"40'119","reported_seq":132,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:56.430324+0000","last_change":"2024-04-23T01:50:40.440701+0000","last_active":"2024-04-23T01:50:56.430324+0000","last_peered":"2024-04-23T01:50:56.430324+0000","last_clean":"2024-04-23T01:50:56.430324+0000","last_became_active":"2024-04-23T01:50:40.433832+0000","last_became_peered":"2024-04-23T01:50:40.433832+0000","last_unstale":"2024-04-23T01:50:56.430324+0000","last_undegraded":"2024-04-23T01:50:56.430324+0000","last_fullsized":"2024-04-23T01:50:56.430324+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"40'103","reported_seq":116,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:56.397186+0000","last_change":"2024-04-23T01:50:40.477278+0000","last_active":"2024-04-23T01:50:56.397186+0000","last_peered":"2024-04-23T01:50:56.397186+0000","last_clean":"2024-04-23T01:50:56.397186+0000","last_became_active":"2024-04-23T01:50:40.471274+0000","last_became_peered":"2024-04-23T01:50:40.471274+0000","last_unstale":"2024-04-23T01:50:56.397186+0000","last_undegraded":"2024-04-23T01:50:56.397186+0000","last_fullsized":"2024-04-23T01:50:56.397186+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"40'175","reported_seq":188,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:56.015570+0000","last_change":"2024-04-23T01:50:40.446224+0000","last_active":"2024-04-23T01:50:56.015570+0000","last_peered":"2024-04-23T01:50:56.015570+0000","last_clean":"2024-04-23T01:50:56.015570+0000","last_became_active":"2024-04-23T01:50:40.437838+0000","last_became_peered":"2024-04-23T01:50:40.437838+0000","last_unstale":"2024-04-23T01:50:56.015570+0000","last_undegraded":"2024-04-23T01:50:56.015570+0000","last_fullsized":"2024-04-23T01:50:56.015570+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"40'111","reported_seq":124,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:56.536680+0000","last_change":"2024-04-23T01:50:40.517607+0000","last_active":"2024-04-23T01:50:56.536680+0000","last_peered":"2024-04-23T01:50:56.536680+0000","last_clean":"2024-04-23T01:50:56.536680+0000","last_became_active":"2024-04-23T01:50:40.511881+0000","last_became_peered":"2024-04-23T01:50:40.511881+0000","last_unstale":"2024-04-23T01:50:56.536680+0000","last_undegraded":"2024-04-23T01:50:56.536680+0000","last_fullsized":"2024-04-23T01:50:56.536680+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"40'188","reported_seq":201,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:56.604860+0000","last_change":"2024-04-23T01:50:40.432817+0000","last_active":"2024-04-23T01:50:56.604860+0000","last_peered":"2024-04-23T01:50:56.604860+0000","last_clean":"2024-04-23T01:50:56.604860+0000","last_became_active":"2024-04-23T01:50:40.421722+0000","last_became_peered":"2024-04-23T01:50:40.421722+0000","last_unstale":"2024-04-23T01:50:56.604860+0000","last_undegraded":"2024-04-23T01:50:56.604860+0000","last_fullsized":"2024-04-23T01:50:56.604860+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548288,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":189,"num_write_kb":1512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"40'79","reported_seq":92,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:53.856879+0000","last_change":"2024-04-23T01:50:40.444510+0000","last_active":"2024-04-23T01:50:53.856879+0000","last_peered":"2024-04-23T01:50:53.856879+0000","last_clean":"2024-04-23T01:50:53.856879+0000","last_became_active":"2024-04-23T01:50:40.437088+0000","last_became_peered":"2024-04-23T01:50:40.437088+0000","last_unstale":"2024-04-23T01:50:53.856879+0000","last_undegraded":"2024-04-23T01:50:53.856879+0000","last_fullsized":"2024-04-23T01:50:53.856879+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"40'215","reported_seq":228,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:55.922585+0000","last_change":"2024-04-23T01:50:40.468123+0000","last_active":"2024-04-23T01:50:55.922585+0000","last_peered":"2024-04-23T01:50:55.922585+0000","last_clean":"2024-04-23T01:50:55.922585+0000","last_became_active":"2024-04-23T01:50:40.443838+0000","last_became_peered":"2024-04-23T01:50:40.443838+0000","last_unstale":"2024-04-23T01:50:55.922585+0000","last_undegraded":"2024-04-23T01:50:55.922585+0000","last_fullsized":"2024-04-23T01:50:55.922585+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1769472,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":216,"num_write_kb":1728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"40'119","reported_seq":132,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:55.503854+0000","last_change":"2024-04-23T01:50:40.451743+0000","last_active":"2024-04-23T01:50:55.503854+0000","last_peered":"2024-04-23T01:50:55.503854+0000","last_clean":"2024-04-23T01:50:55.503854+0000","last_became_active":"2024-04-23T01:50:40.447092+0000","last_became_peered":"2024-04-23T01:50:40.447092+0000","last_unstale":"2024-04-23T01:50:55.503854+0000","last_undegraded":"2024-04-23T01:50:55.503854+0000","last_fullsized":"2024-04-23T01:50:55.503854+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"40'87","reported_seq":100,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:53.956399+0000","last_change":"2024-04-23T01:50:40.477557+0000","last_active":"2024-04-23T01:50:53.956399+0000","last_peered":"2024-04-23T01:50:53.956399+0000","last_clean":"2024-04-23T01:50:53.956399+0000","last_became_active":"2024-04-23T01:50:40.472479+0000","last_became_peered":"2024-04-23T01:50:40.472479+0000","last_unstale":"2024-04-23T01:50:53.956399+0000","last_undegraded":"2024-04-23T01:50:53.956399+0000","last_fullsized":"2024-04-23T01:50:53.956399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":720896,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":215,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:41.626121+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:50:41.626121+0000","last_peered":"2024-04-23T01:50:41.626121+0000","last_clean":"2024-04-23T01:50:41.626121+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:50:41.626121+0000","last_undegraded":"2024-04-23T01:50:41.626121+0000","last_fullsized":"2024-04-23T01:50:41.626121+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":82,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:41.615653+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:50:41.615653+0000","last_peered":"2024-04-23T01:50:41.615653+0000","last_clean":"2024-04-23T01:50:41.615653+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:50:41.615653+0000","last_undegraded":"2024-04-23T01:50:41.615653+0000","last_fullsized":"2024-04-23T01:50:41.615653+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"40'112","reported_seq":125,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:56.187563+0000","last_change":"2024-04-23T01:50:40.431365+0000","last_active":"2024-04-23T01:50:56.187563+0000","last_peered":"2024-04-23T01:50:56.187563+0000","last_clean":"2024-04-23T01:50:56.187563+0000","last_became_active":"2024-04-23T01:50:40.426237+0000","last_became_peered":"2024-04-23T01:50:40.426237+0000","last_unstale":"2024-04-23T01:50:56.187563+0000","last_undegraded":"2024-04-23T01:50:56.187563+0000","last_fullsized":"2024-04-23T01:50:56.187563+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":925696,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113,"num_write_kb":904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":80,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:41.588811+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:50:41.588811+0000","last_peered":"2024-04-23T01:50:41.588811+0000","last_clean":"2024-04-23T01:50:41.588811+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:50:41.588811+0000","last_undegraded":"2024-04-23T01:50:41.588811+0000","last_fullsized":"2024-04-23T01:50:41.588811+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"40'135","reported_seq":148,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:54.442282+0000","last_change":"2024-04-23T01:50:40.486856+0000","last_active":"2024-04-23T01:50:54.442282+0000","last_peered":"2024-04-23T01:50:54.442282+0000","last_clean":"2024-04-23T01:50:54.442282+0000","last_became_active":"2024-04-23T01:50:40.478453+0000","last_became_peered":"2024-04-23T01:50:40.478453+0000","last_unstale":"2024-04-23T01:50:54.442282+0000","last_undegraded":"2024-04-23T01:50:54.442282+0000","last_fullsized":"2024-04-23T01:50:54.442282+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":136,"num_write_kb":1088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":82,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:41.608295+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:50:41.608295+0000","last_peered":"2024-04-23T01:50:41.608295+0000","last_clean":"2024-04-23T01:50:41.608295+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:50:41.608295+0000","last_undegraded":"2024-04-23T01:50:41.608295+0000","last_fullsized":"2024-04-23T01:50:41.608295+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"40'151","reported_seq":164,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:56.075479+0000","last_change":"2024-04-23T01:50:40.462462+0000","last_active":"2024-04-23T01:50:56.075479+0000","last_peered":"2024-04-23T01:50:56.075479+0000","last_clean":"2024-04-23T01:50:56.075479+0000","last_became_active":"2024-04-23T01:50:40.439584+0000","last_became_peered":"2024-04-23T01:50:40.439584+0000","last_unstale":"2024-04-23T01:50:56.075479+0000","last_undegraded":"2024-04-23T01:50:56.075479+0000","last_fullsized":"2024-04-23T01:50:56.075479+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":80,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:41.597584+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:50:41.597584+0000","last_peered":"2024-04-23T01:50:41.597584+0000","last_clean":"2024-04-23T01:50:41.597584+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:50:41.597584+0000","last_undegraded":"2024-04-23T01:50:41.597584+0000","last_fullsized":"2024-04-23T01:50:41.597584+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"40'119","reported_seq":132,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:55.731884+0000","last_change":"2024-04-23T01:50:40.466468+0000","last_active":"2024-04-23T01:50:55.731884+0000","last_peered":"2024-04-23T01:50:55.731884+0000","last_clean":"2024-04-23T01:50:55.731884+0000","last_became_active":"2024-04-23T01:50:40.452408+0000","last_became_peered":"2024-04-23T01:50:40.452408+0000","last_unstale":"2024-04-23T01:50:55.731884+0000","last_undegraded":"2024-04-23T01:50:55.731884+0000","last_fullsized":"2024-04-23T01:50:55.731884+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":86,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:41.290654+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:50:41.290654+0000","last_peered":"2024-04-23T01:50:41.290654+0000","last_clean":"2024-04-23T01:50:41.290654+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:50:41.290654+0000","last_undegraded":"2024-04-23T01:50:41.290654+0000","last_fullsized":"2024-04-23T01:50:41.290654+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"40'196","reported_seq":209,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:56.548978+0000","last_change":"2024-04-23T01:50:40.469709+0000","last_active":"2024-04-23T01:50:56.548978+0000","last_peered":"2024-04-23T01:50:56.548978+0000","last_clean":"2024-04-23T01:50:56.548978+0000","last_became_active":"2024-04-23T01:50:40.462062+0000","last_became_peered":"2024-04-23T01:50:40.462062+0000","last_unstale":"2024-04-23T01:50:56.548978+0000","last_undegraded":"2024-04-23T01:50:56.548978+0000","last_fullsized":"2024-04-23T01:50:56.548978+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613824,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":197,"num_write_kb":1576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":82,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:41.626809+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:50:41.626809+0000","last_peered":"2024-04-23T01:50:41.626809+0000","last_clean":"2024-04-23T01:50:41.626809+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:50:41.626809+0000","last_undegraded":"2024-04-23T01:50:41.626809+0000","last_fullsized":"2024-04-23T01:50:41.626809+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"40'191","reported_seq":204,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:56.553707+0000","last_change":"2024-04-23T01:50:40.464715+0000","last_active":"2024-04-23T01:50:56.553707+0000","last_peered":"2024-04-23T01:50:56.553707+0000","last_clean":"2024-04-23T01:50:56.553707+0000","last_became_active":"2024-04-23T01:50:40.457999+0000","last_became_peered":"2024-04-23T01:50:40.457999+0000","last_unstale":"2024-04-23T01:50:56.553707+0000","last_undegraded":"2024-04-23T01:50:56.553707+0000","last_fullsized":"2024-04-23T01:50:56.553707+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1572864,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":192,"num_write_kb":1536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":86,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:41.311541+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:50:41.311541+0000","last_peered":"2024-04-23T01:50:41.311541+0000","last_clean":"2024-04-23T01:50:41.311541+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:50:41.311541+0000","last_undegraded":"2024-04-23T01:50:41.311541+0000","last_fullsized":"2024-04-23T01:50:41.311541+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"40'223","reported_seq":236,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:56.397389+0000","last_change":"2024-04-23T01:50:40.447151+0000","last_active":"2024-04-23T01:50:56.397389+0000","last_peered":"2024-04-23T01:50:56.397389+0000","last_clean":"2024-04-23T01:50:56.397389+0000","last_became_active":"2024-04-23T01:50:40.441283+0000","last_became_peered":"2024-04-23T01:50:40.441283+0000","last_unstale":"2024-04-23T01:50:56.397389+0000","last_undegraded":"2024-04-23T01:50:56.397389+0000","last_fullsized":"2024-04-23T01:50:56.397389+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1835008,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":1792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":82,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:41.624618+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:50:41.624618+0000","last_peered":"2024-04-23T01:50:41.624618+0000","last_clean":"2024-04-23T01:50:41.624618+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:50:41.624618+0000","last_undegraded":"2024-04-23T01:50:41.624618+0000","last_fullsized":"2024-04-23T01:50:41.624618+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":19161088,"num_objects":294,"num_object_clones":0,"num_object_copies":882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2339,"num_write_kb":18712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28753920,"data_stored":57507840,"data_compressed":2966130,"data_compressed_allocated":28753920,"data_compressed_original":57507840,"omap_allocated":0,"internal_metadata":0},"log_size":2323,"ondisk_log_size":2323,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":69676,"kb_used_data":6752,"kb_used_omap":0,"kb_used_meta":62912,"kb_avail":94302164,"statfs":{"total":96636764160,"available":96565415936,"internally_reserved":0,"allocated":6914048,"data_stored":13726365,"data_compressed":701298,"data_compressed_allocated":6549504,"data_compressed_original":13590528,"omap_allocated":0,"internal_metadata":64421888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":73588,"kb_used_data":6508,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94298252,"statfs":{"total":96636764160,"available":96561410048,"internally_reserved":0,"allocated":6664192,"data_stored":13226653,"data_compressed":675555,"data_compressed_allocated":6299648,"data_compressed_original":13090816,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":79088,"kb_used_data":7848,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94292752,"statfs":{"total":96636764160,"available":96555778048,"internally_reserved":0,"allocated":8036352,"data_stored":15503485,"data_compressed":793470,"data_compressed_allocated":7692288,"data_compressed_original":15384576,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":78708,"kb_used_data":7464,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94293132,"statfs":{"total":96636764160,"available":96556167168,"internally_reserved":0,"allocated":7643136,"data_stored":15184541,"data_compressed":776532,"data_compressed_allocated":7278592,"data_compressed_original":15048704,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7467008,"data_stored":14934016,"data_compressed":770267,"data_compressed_allocated":7467008,"data_compressed_original":14934016,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8052736,"data_stored":16105472,"data_compressed":830694,"data_compressed_allocated":8052736,"data_compressed_original":16105472,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6492160,"data_stored":12984320,"data_compressed":669713,"data_compressed_allocated":6492160,"data_compressed_original":12984320,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6742016,"data_stored":13484032,"data_compressed":695456,"data_compressed_allocated":6742016,"data_compressed_original":13484032,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:50:59.352 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 18 16 2759 2743 1.19044 1.23438 0.128734 0.104689 2024-04-23T01:50:59.641 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:50:59.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:59.772+0000 7fe84682d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:50:59.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:50:59.779+0000 7fe84682d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:00.352 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 19 16 2912 2896 1.19069 1.19531 0.13117 0.10468 2024-04-23T01:51:01.352 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:51:01.353078+0000 min lat: 0.0144892 max lat: 0.287285 avg lat: 0.104597 2024-04-23T01:51:01.352 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:51:01.352 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 20 16 3066 3050 1.1913 1.20312 0.0949204 0.104597 2024-04-23T01:51:01.958 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:51:02.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:02.093+0000 7f19ddd5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:02.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:02.101+0000 7f19ddd5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:02.352 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 21 16 3203 3187 1.18554 1.07031 0.0973759 0.105034 2024-04-23T01:51:03.352 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 22 16 3354 3338 1.18527 1.17969 0.0801067 0.105012 2024-04-23T01:51:03.573 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:51:03.573 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:51:03.573 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:51:03.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:03.691+0000 7f6e7e206640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:03.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:03.700+0000 7f6e7e206640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:03.906 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:51:03.907 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:51:03.923 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-04-23T01:51:03.617472+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":25903648,"num_objects":390,"num_object_clones":0,"num_object_copies":1170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":3227,"num_write_kb":26614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3171,"ondisk_log_size":3171,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":318440,"kb_used_data":37636,"kb_used_omap":0,"kb_used_meta":280768,"kb_avail":377168920,"statfs":{"total":386547056640,"available":386220974080,"internally_reserved":0,"allocated":38539264,"data_stored":76204116,"data_compressed":3905373,"data_compressed_allocated":37101568,"data_compressed_original":75677696,"omap_allocated":0,"internal_metadata":287506432},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18685952,"num_objects":286,"num_object_clones":0,"num_object_copies":858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2281,"num_write_kb":18248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002137"},"pg_stats":[{"pgid":"6.f","version":"40'167","reported_seq":180,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:51:00.371439+0000","last_change":"2024-04-23T01:50:40.440701+0000","last_active":"2024-04-23T01:51:00.371439+0000","last_peered":"2024-04-23T01:51:00.371439+0000","last_clean":"2024-04-23T01:51:00.371439+0000","last_became_active":"2024-04-23T01:50:40.433832+0000","last_became_peered":"2024-04-23T01:50:40.433832+0000","last_unstale":"2024-04-23T01:51:00.371439+0000","last_undegraded":"2024-04-23T01:51:00.371439+0000","last_fullsized":"2024-04-23T01:51:00.371439+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":1344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"40'175","reported_seq":188,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:51:01.463679+0000","last_change":"2024-04-23T01:50:40.477278+0000","last_active":"2024-04-23T01:51:01.463679+0000","last_peered":"2024-04-23T01:51:01.463679+0000","last_clean":"2024-04-23T01:51:01.463679+0000","last_became_active":"2024-04-23T01:50:40.471274+0000","last_became_peered":"2024-04-23T01:50:40.471274+0000","last_unstale":"2024-04-23T01:51:01.463679+0000","last_undegraded":"2024-04-23T01:51:01.463679+0000","last_fullsized":"2024-04-23T01:51:01.463679+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"40'221","reported_seq":234,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:51:01.594834+0000","last_change":"2024-04-23T01:50:40.446224+0000","last_active":"2024-04-23T01:51:01.594834+0000","last_peered":"2024-04-23T01:51:01.594834+0000","last_clean":"2024-04-23T01:51:01.594834+0000","last_became_active":"2024-04-23T01:50:40.437838+0000","last_became_peered":"2024-04-23T01:50:40.437838+0000","last_unstale":"2024-04-23T01:51:01.594834+0000","last_undegraded":"2024-04-23T01:51:01.594834+0000","last_fullsized":"2024-04-23T01:51:01.594834+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1818624,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":222,"num_write_kb":1776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"40'151","reported_seq":164,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:51:00.867517+0000","last_change":"2024-04-23T01:50:40.517607+0000","last_active":"2024-04-23T01:51:00.867517+0000","last_peered":"2024-04-23T01:51:00.867517+0000","last_clean":"2024-04-23T01:51:00.867517+0000","last_became_active":"2024-04-23T01:50:40.511881+0000","last_became_peered":"2024-04-23T01:50:40.511881+0000","last_unstale":"2024-04-23T01:51:00.867517+0000","last_undegraded":"2024-04-23T01:51:00.867517+0000","last_fullsized":"2024-04-23T01:51:00.867517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"40'231","reported_seq":244,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:59.997576+0000","last_change":"2024-04-23T01:50:40.432817+0000","last_active":"2024-04-23T01:50:59.997576+0000","last_peered":"2024-04-23T01:50:59.997576+0000","last_clean":"2024-04-23T01:50:59.997576+0000","last_became_active":"2024-04-23T01:50:40.421722+0000","last_became_peered":"2024-04-23T01:50:40.421722+0000","last_unstale":"2024-04-23T01:50:59.997576+0000","last_undegraded":"2024-04-23T01:50:59.997576+0000","last_fullsized":"2024-04-23T01:50:59.997576+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1900544,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":232,"num_write_kb":1856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"40'119","reported_seq":132,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:51:00.440098+0000","last_change":"2024-04-23T01:50:40.444510+0000","last_active":"2024-04-23T01:51:00.440098+0000","last_peered":"2024-04-23T01:51:00.440098+0000","last_clean":"2024-04-23T01:51:00.440098+0000","last_became_active":"2024-04-23T01:50:40.437088+0000","last_became_peered":"2024-04-23T01:50:40.437088+0000","last_unstale":"2024-04-23T01:51:00.440098+0000","last_undegraded":"2024-04-23T01:51:00.440098+0000","last_fullsized":"2024-04-23T01:51:00.440098+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"40'262","reported_seq":275,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:51:01.593944+0000","last_change":"2024-04-23T01:50:40.468123+0000","last_active":"2024-04-23T01:51:01.593944+0000","last_peered":"2024-04-23T01:51:01.593944+0000","last_clean":"2024-04-23T01:51:01.593944+0000","last_became_active":"2024-04-23T01:50:40.443838+0000","last_became_peered":"2024-04-23T01:50:40.443838+0000","last_unstale":"2024-04-23T01:51:01.593944+0000","last_undegraded":"2024-04-23T01:51:01.593944+0000","last_fullsized":"2024-04-23T01:51:01.593944+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2154496,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":263,"num_write_kb":2104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"40'143","reported_seq":156,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:51:01.454126+0000","last_change":"2024-04-23T01:50:40.451743+0000","last_active":"2024-04-23T01:51:01.454126+0000","last_peered":"2024-04-23T01:51:01.454126+0000","last_clean":"2024-04-23T01:51:01.454126+0000","last_became_active":"2024-04-23T01:50:40.447092+0000","last_became_peered":"2024-04-23T01:50:40.447092+0000","last_unstale":"2024-04-23T01:51:01.454126+0000","last_undegraded":"2024-04-23T01:51:01.454126+0000","last_fullsized":"2024-04-23T01:51:01.454126+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"40'127","reported_seq":140,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:51:01.337876+0000","last_change":"2024-04-23T01:50:40.477557+0000","last_active":"2024-04-23T01:51:01.337876+0000","last_peered":"2024-04-23T01:51:01.337876+0000","last_clean":"2024-04-23T01:51:01.337876+0000","last_became_active":"2024-04-23T01:50:40.472479+0000","last_became_peered":"2024-04-23T01:50:40.472479+0000","last_unstale":"2024-04-23T01:51:01.337876+0000","last_undegraded":"2024-04-23T01:51:01.337876+0000","last_fullsized":"2024-04-23T01:51:01.337876+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":215,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:41.626121+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:50:41.626121+0000","last_peered":"2024-04-23T01:50:41.626121+0000","last_clean":"2024-04-23T01:50:41.626121+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:50:41.626121+0000","last_undegraded":"2024-04-23T01:50:41.626121+0000","last_fullsized":"2024-04-23T01:50:41.626121+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":82,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:41.615653+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:50:41.615653+0000","last_peered":"2024-04-23T01:50:41.615653+0000","last_clean":"2024-04-23T01:50:41.615653+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:50:41.615653+0000","last_undegraded":"2024-04-23T01:50:41.615653+0000","last_fullsized":"2024-04-23T01:50:41.615653+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"40'143","reported_seq":156,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:51:00.211967+0000","last_change":"2024-04-23T01:50:40.431365+0000","last_active":"2024-04-23T01:51:00.211967+0000","last_peered":"2024-04-23T01:51:00.211967+0000","last_clean":"2024-04-23T01:51:00.211967+0000","last_became_active":"2024-04-23T01:50:40.426237+0000","last_became_peered":"2024-04-23T01:50:40.426237+0000","last_unstale":"2024-04-23T01:51:00.211967+0000","last_undegraded":"2024-04-23T01:51:00.211967+0000","last_fullsized":"2024-04-23T01:51:00.211967+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":80,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:41.588811+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:50:41.588811+0000","last_peered":"2024-04-23T01:50:41.588811+0000","last_clean":"2024-04-23T01:50:41.588811+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:50:41.588811+0000","last_undegraded":"2024-04-23T01:50:41.588811+0000","last_fullsized":"2024-04-23T01:50:41.588811+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"40'201","reported_seq":214,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:51:01.597157+0000","last_change":"2024-04-23T01:50:40.486856+0000","last_active":"2024-04-23T01:51:01.597157+0000","last_peered":"2024-04-23T01:51:01.597157+0000","last_clean":"2024-04-23T01:51:01.597157+0000","last_became_active":"2024-04-23T01:50:40.478453+0000","last_became_peered":"2024-04-23T01:50:40.478453+0000","last_unstale":"2024-04-23T01:51:01.597157+0000","last_undegraded":"2024-04-23T01:51:01.597157+0000","last_fullsized":"2024-04-23T01:51:01.597157+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1654784,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":202,"num_write_kb":1616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":82,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:41.608295+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:50:41.608295+0000","last_peered":"2024-04-23T01:50:41.608295+0000","last_clean":"2024-04-23T01:50:41.608295+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:50:41.608295+0000","last_undegraded":"2024-04-23T01:50:41.608295+0000","last_fullsized":"2024-04-23T01:50:41.608295+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"40'207","reported_seq":220,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:51:00.661941+0000","last_change":"2024-04-23T01:50:40.462462+0000","last_active":"2024-04-23T01:51:00.661941+0000","last_peered":"2024-04-23T01:51:00.661941+0000","last_clean":"2024-04-23T01:51:00.661941+0000","last_became_active":"2024-04-23T01:50:40.439584+0000","last_became_peered":"2024-04-23T01:50:40.439584+0000","last_unstale":"2024-04-23T01:51:00.661941+0000","last_undegraded":"2024-04-23T01:51:00.661941+0000","last_fullsized":"2024-04-23T01:51:00.661941+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703936,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":208,"num_write_kb":1664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":80,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:41.597584+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:50:41.597584+0000","last_peered":"2024-04-23T01:50:41.597584+0000","last_clean":"2024-04-23T01:50:41.597584+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:50:41.597584+0000","last_undegraded":"2024-04-23T01:50:41.597584+0000","last_fullsized":"2024-04-23T01:50:41.597584+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"40'183","reported_seq":196,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:51:01.389110+0000","last_change":"2024-04-23T01:50:40.466468+0000","last_active":"2024-04-23T01:51:01.389110+0000","last_peered":"2024-04-23T01:51:01.389110+0000","last_clean":"2024-04-23T01:51:01.389110+0000","last_became_active":"2024-04-23T01:50:40.452408+0000","last_became_peered":"2024-04-23T01:50:40.452408+0000","last_unstale":"2024-04-23T01:51:01.389110+0000","last_undegraded":"2024-04-23T01:51:01.389110+0000","last_fullsized":"2024-04-23T01:51:01.389110+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":86,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:41.290654+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:50:41.290654+0000","last_peered":"2024-04-23T01:50:41.290654+0000","last_clean":"2024-04-23T01:50:41.290654+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:50:41.290654+0000","last_undegraded":"2024-04-23T01:50:41.290654+0000","last_fullsized":"2024-04-23T01:50:41.290654+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"40'255","reported_seq":268,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:51:01.059080+0000","last_change":"2024-04-23T01:50:40.469709+0000","last_active":"2024-04-23T01:51:01.059080+0000","last_peered":"2024-04-23T01:51:01.059080+0000","last_clean":"2024-04-23T01:51:01.059080+0000","last_became_active":"2024-04-23T01:50:40.462062+0000","last_became_peered":"2024-04-23T01:50:40.462062+0000","last_unstale":"2024-04-23T01:51:01.059080+0000","last_undegraded":"2024-04-23T01:51:01.059080+0000","last_fullsized":"2024-04-23T01:51:01.059080+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":32,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":256,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":82,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:41.626809+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:50:41.626809+0000","last_peered":"2024-04-23T01:50:41.626809+0000","last_clean":"2024-04-23T01:50:41.626809+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:50:41.626809+0000","last_undegraded":"2024-04-23T01:50:41.626809+0000","last_fullsized":"2024-04-23T01:50:41.626809+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"40'223","reported_seq":236,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:51:01.126490+0000","last_change":"2024-04-23T01:50:40.464715+0000","last_active":"2024-04-23T01:51:01.126490+0000","last_peered":"2024-04-23T01:51:01.126490+0000","last_clean":"2024-04-23T01:51:01.126490+0000","last_became_active":"2024-04-23T01:50:40.457999+0000","last_became_peered":"2024-04-23T01:50:40.457999+0000","last_unstale":"2024-04-23T01:51:01.126490+0000","last_undegraded":"2024-04-23T01:51:01.126490+0000","last_fullsized":"2024-04-23T01:51:01.126490+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1835008,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":1792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":86,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:41.311541+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:50:41.311541+0000","last_peered":"2024-04-23T01:50:41.311541+0000","last_clean":"2024-04-23T01:50:41.311541+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:50:41.311541+0000","last_undegraded":"2024-04-23T01:50:41.311541+0000","last_fullsized":"2024-04-23T01:50:41.311541+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"40'266","reported_seq":279,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:51:01.594556+0000","last_change":"2024-04-23T01:50:40.447151+0000","last_active":"2024-04-23T01:51:01.594556+0000","last_peered":"2024-04-23T01:51:01.594556+0000","last_clean":"2024-04-23T01:51:01.594556+0000","last_became_active":"2024-04-23T01:50:40.441283+0000","last_became_peered":"2024-04-23T01:50:40.441283+0000","last_unstale":"2024-04-23T01:51:01.594556+0000","last_undegraded":"2024-04-23T01:51:01.594556+0000","last_fullsized":"2024-04-23T01:51:01.594556+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2187264,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":267,"num_write_kb":2136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":82,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-23T01:50:41.624618+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:50:41.624618+0000","last_peered":"2024-04-23T01:50:41.624618+0000","last_clean":"2024-04-23T01:50:41.624618+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:50:41.624618+0000","last_undegraded":"2024-04-23T01:50:41.624618+0000","last_fullsized":"2024-04-23T01:50:41.624618+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":25313280,"num_objects":388,"num_object_clones":0,"num_object_copies":1164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3090,"num_write_kb":24720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":37969920,"data_stored":75939840,"data_compressed":3917880,"data_compressed_allocated":37969920,"data_compressed_original":75939840,"omap_allocated":0,"internal_metadata":0},"log_size":3074,"ondisk_log_size":3074,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":71652,"kb_used_data":8732,"kb_used_omap":0,"kb_used_meta":62912,"kb_avail":94300188,"statfs":{"total":96636764160,"available":96563392512,"internally_reserved":0,"allocated":8941568,"data_stored":17781405,"data_compressed":910683,"data_compressed_allocated":8577024,"data_compressed_original":17645568,"omap_allocated":0,"internal_metadata":64421888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":79920,"kb_used_data":8680,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94291920,"statfs":{"total":96636764160,"available":96554926080,"internally_reserved":0,"allocated":8888320,"data_stored":17674909,"data_compressed":905244,"data_compressed_allocated":8523776,"data_compressed_original":17539072,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":81564,"kb_used_data":10324,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94290276,"statfs":{"total":96636764160,"available":96553242624,"internally_reserved":0,"allocated":10571776,"data_stored":20574333,"data_compressed":1055307,"data_compressed_allocated":10227712,"data_compressed_original":20455424,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":85304,"kb_used_data":9900,"kb_used_omap":0,"kb_used_meta":75392,"kb_avail":94286536,"statfs":{"total":96636764160,"available":96549412864,"internally_reserved":0,"allocated":10137600,"data_stored":20173469,"data_compressed":1034139,"data_compressed_allocated":9773056,"data_compressed_original":20037632,"omap_allocated":0,"internal_metadata":77201408},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":10010624,"data_stored":20021248,"data_compressed":1032950,"data_compressed_allocated":10010624,"data_compressed_original":20021248,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10493952,"data_stored":20987904,"data_compressed":1082802,"data_compressed_allocated":10493952,"data_compressed_original":20987904,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8683520,"data_stored":17367040,"data_compressed":896018,"data_compressed_allocated":8683520,"data_compressed_original":17367040,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8781824,"data_stored":17563648,"data_compressed":906110,"data_compressed_allocated":8781824,"data_compressed_original":17563648,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:51:04.311 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:51:04.352 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 23 16 3501 3485 1.18366 1.14844 0.208908 0.105253 2024-04-23T01:51:04.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:04.430+0000 7fcb182ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:04.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:04.437+0000 7fcb182ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:05.352 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 24 16 3656 3640 1.18479 1.21094 0.136583 0.105257 2024-04-23T01:51:06.353 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 25 16 3820 3804 1.18865 1.28125 0.0827317 0.104948 2024-04-23T01:51:06.638 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:51:06.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:06.769+0000 7f7dd499d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:06.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:06.776+0000 7f7dd499d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:07.353 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 26 16 3979 3963 1.1907 1.24219 0.0893251 0.104685 2024-04-23T01:51:08.353 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 27 16 4123 4107 1.18827 1.125 0.0944958 0.104863 2024-04-23T01:51:08.925 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:51:08.926 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-23T01:51:08.940 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:51:09.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:09.047+0000 7f5d1c2ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:09.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:09.061+0000 7f5d1c2ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:09.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:09.107+0000 7efef30ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:09.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:09.115+0000 7efef30ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:09.353 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 28 16 4262 4246 1.18461 1.08594 0.18211 0.10517 2024-04-23T01:51:10.323 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.0 to 1 (10000) 2024-04-23T01:51:10.338 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-23T01:51:10.353 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 29 16 4416 4400 1.18524 1.20312 0.0616616 0.105202 2024-04-23T01:51:10.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:10.460+0000 7f2b0a26d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:10.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:10.472+0000 7f2b0a26d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:11.328 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.1 to 1 (10000) 2024-04-23T01:51:11.344 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-23T01:51:11.353 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 30 16 4538 4522 1.1775 0.953125 0.158979 0.10593 2024-04-23T01:51:11.371 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:51:11.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:11.492+0000 7f05cf510640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:11.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:11.508+0000 7f05cf510640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:11.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:11.531+0000 7fa03395d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:11.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:11.538+0000 7fa03395d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:12.337 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.2 to 1 (10000) 2024-04-23T01:51:12.353 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 31 16 4674 4658 1.17379 1.0625 0.157996 0.106294 2024-04-23T01:51:12.355 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-23T01:51:12.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:12.495+0000 7f641fd9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:12.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:12.507+0000 7f641fd9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:13.351 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.3 to 1 (10000) 2024-04-23T01:51:13.353 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 32 16 4817 4801 1.17202 1.11719 0.0770057 0.106447 2024-04-23T01:51:13.370 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-23T01:51:13.370 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:51:13.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:13.497+0000 7f152f46d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:13.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:13.509+0000 7f152f46d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:13.704 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:51:13.760 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:51:13.761 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:51:13.777 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-04-23T01:51:13.619383+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":37921312,"num_objects":572,"num_object_clones":0,"num_object_copies":1716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":4694,"num_write_kb":38350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4650,"ondisk_log_size":4650,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":336104,"kb_used_data":55304,"kb_used_omap":0,"kb_used_meta":280768,"kb_avail":377151256,"statfs":{"total":386547056640,"available":386202886144,"internally_reserved":0,"allocated":56631296,"data_stored":112335624,"data_compressed":5770380,"data_compressed_allocated":55160832,"data_compressed_original":111796224,"omap_allocated":0,"internal_metadata":287506432},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5808128,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":709,"num_write_kb":5672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000637"},"pg_stats":[{"pgid":"6.f","version":"42'284","reported_seq":303,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.606123+0000","last_change":"2024-04-23T01:50:40.440701+0000","last_active":"2024-04-23T01:51:11.606123+0000","last_peered":"2024-04-23T01:51:11.606123+0000","last_clean":"2024-04-23T01:51:11.606123+0000","last_became_active":"2024-04-23T01:50:40.433832+0000","last_became_peered":"2024-04-23T01:50:40.433832+0000","last_unstale":"2024-04-23T01:51:11.606123+0000","last_undegraded":"2024-04-23T01:51:11.606123+0000","last_fullsized":"2024-04-23T01:51:11.606123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2334720,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":285,"num_write_kb":2280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"41'256","reported_seq":274,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.635941+0000","last_change":"2024-04-23T01:50:40.477278+0000","last_active":"2024-04-23T01:51:11.635941+0000","last_peered":"2024-04-23T01:51:11.635941+0000","last_clean":"2024-04-23T01:51:11.635941+0000","last_became_active":"2024-04-23T01:50:40.471274+0000","last_became_peered":"2024-04-23T01:50:40.471274+0000","last_unstale":"2024-04-23T01:51:11.635941+0000","last_undegraded":"2024-04-23T01:51:11.635941+0000","last_fullsized":"2024-04-23T01:51:11.635941+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":32,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":256,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"41'304","reported_seq":322,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.624878+0000","last_change":"2024-04-23T01:50:40.446224+0000","last_active":"2024-04-23T01:51:11.624878+0000","last_peered":"2024-04-23T01:51:11.624878+0000","last_clean":"2024-04-23T01:51:11.624878+0000","last_became_active":"2024-04-23T01:50:40.437838+0000","last_became_peered":"2024-04-23T01:50:40.437838+0000","last_unstale":"2024-04-23T01:51:11.624878+0000","last_undegraded":"2024-04-23T01:51:11.624878+0000","last_fullsized":"2024-04-23T01:51:11.624878+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490368,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"40'208","reported_seq":223,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-23T01:51:10.818987+0000","last_change":"2024-04-23T01:50:40.517607+0000","last_active":"2024-04-23T01:51:10.818987+0000","last_peered":"2024-04-23T01:51:10.818987+0000","last_clean":"2024-04-23T01:51:10.818987+0000","last_became_active":"2024-04-23T01:50:40.511881+0000","last_became_peered":"2024-04-23T01:50:40.511881+0000","last_unstale":"2024-04-23T01:51:10.818987+0000","last_undegraded":"2024-04-23T01:51:10.818987+0000","last_fullsized":"2024-04-23T01:51:10.818987+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703936,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":208,"num_write_kb":1664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"41'352","reported_seq":370,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.630159+0000","last_change":"2024-04-23T01:50:40.432817+0000","last_active":"2024-04-23T01:51:11.630159+0000","last_peered":"2024-04-23T01:51:11.630159+0000","last_clean":"2024-04-23T01:51:11.630159+0000","last_became_active":"2024-04-23T01:50:40.421722+0000","last_became_peered":"2024-04-23T01:50:40.421722+0000","last_unstale":"2024-04-23T01:51:11.630159+0000","last_undegraded":"2024-04-23T01:51:11.630159+0000","last_fullsized":"2024-04-23T01:51:11.630159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883584,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":352,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"42'183","reported_seq":202,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.542547+0000","last_change":"2024-04-23T01:50:40.444510+0000","last_active":"2024-04-23T01:51:11.542547+0000","last_peered":"2024-04-23T01:51:11.542547+0000","last_clean":"2024-04-23T01:51:11.542547+0000","last_became_active":"2024-04-23T01:50:40.437088+0000","last_became_peered":"2024-04-23T01:50:40.437088+0000","last_unstale":"2024-04-23T01:51:11.542547+0000","last_undegraded":"2024-04-23T01:51:11.542547+0000","last_fullsized":"2024-04-23T01:51:11.542547+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"40'320","reported_seq":335,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-23T01:51:10.818793+0000","last_change":"2024-04-23T01:50:40.468123+0000","last_active":"2024-04-23T01:51:10.818793+0000","last_peered":"2024-04-23T01:51:10.818793+0000","last_clean":"2024-04-23T01:51:10.818793+0000","last_became_active":"2024-04-23T01:50:40.443838+0000","last_became_peered":"2024-04-23T01:50:40.443838+0000","last_unstale":"2024-04-23T01:51:10.818793+0000","last_undegraded":"2024-04-23T01:51:10.818793+0000","last_fullsized":"2024-04-23T01:51:10.818793+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":320,"num_write_kb":2560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"41'263","reported_seq":279,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.543637+0000","last_change":"2024-04-23T01:50:40.451743+0000","last_active":"2024-04-23T01:51:11.543637+0000","last_peered":"2024-04-23T01:51:11.543637+0000","last_clean":"2024-04-23T01:51:11.543637+0000","last_became_active":"2024-04-23T01:50:40.447092+0000","last_became_peered":"2024-04-23T01:50:40.447092+0000","last_unstale":"2024-04-23T01:51:11.543637+0000","last_undegraded":"2024-04-23T01:51:11.543637+0000","last_fullsized":"2024-04-23T01:51:11.543637+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162688,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":264,"num_write_kb":2112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"40'248","reported_seq":263,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:10.806777+0000","last_change":"2024-04-23T01:50:40.477557+0000","last_active":"2024-04-23T01:51:10.806777+0000","last_peered":"2024-04-23T01:51:10.806777+0000","last_clean":"2024-04-23T01:51:10.806777+0000","last_became_active":"2024-04-23T01:50:40.472479+0000","last_became_peered":"2024-04-23T01:50:40.472479+0000","last_unstale":"2024-04-23T01:51:10.806777+0000","last_undegraded":"2024-04-23T01:51:10.806777+0000","last_fullsized":"2024-04-23T01:51:10.806777+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":218,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-23T01:51:10.817606+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:51:10.817606+0000","last_peered":"2024-04-23T01:51:10.817606+0000","last_clean":"2024-04-23T01:51:10.817606+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:51:10.817606+0000","last_undegraded":"2024-04-23T01:51:10.817606+0000","last_fullsized":"2024-04-23T01:51:10.817606+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":88,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.628370+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:51:11.628370+0000","last_peered":"2024-04-23T01:51:11.628370+0000","last_clean":"2024-04-23T01:51:11.628370+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:51:11.628370+0000","last_undegraded":"2024-04-23T01:51:11.628370+0000","last_fullsized":"2024-04-23T01:51:11.628370+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"41'239","reported_seq":255,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.381514+0000","last_change":"2024-04-23T01:50:40.431365+0000","last_active":"2024-04-23T01:51:11.381514+0000","last_peered":"2024-04-23T01:51:11.381514+0000","last_clean":"2024-04-23T01:51:11.381514+0000","last_became_active":"2024-04-23T01:50:40.426237+0000","last_became_peered":"2024-04-23T01:50:40.426237+0000","last_unstale":"2024-04-23T01:51:11.381514+0000","last_undegraded":"2024-04-23T01:51:11.381514+0000","last_fullsized":"2024-04-23T01:51:11.381514+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":240,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":83,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:10.806611+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:51:10.806611+0000","last_peered":"2024-04-23T01:51:10.806611+0000","last_clean":"2024-04-23T01:51:10.806611+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:51:10.806611+0000","last_undegraded":"2024-04-23T01:51:10.806611+0000","last_fullsized":"2024-04-23T01:51:10.806611+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"41'288","reported_seq":306,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.341280+0000","last_change":"2024-04-23T01:50:40.486856+0000","last_active":"2024-04-23T01:51:11.341280+0000","last_peered":"2024-04-23T01:51:11.341280+0000","last_clean":"2024-04-23T01:51:11.341280+0000","last_became_active":"2024-04-23T01:50:40.478453+0000","last_became_peered":"2024-04-23T01:50:40.478453+0000","last_unstale":"2024-04-23T01:51:11.341280+0000","last_undegraded":"2024-04-23T01:51:11.341280+0000","last_fullsized":"2024-04-23T01:51:11.341280+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":288,"log_dups_size":0,"ondisk_log_size":288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":88,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.631556+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:51:11.631556+0000","last_peered":"2024-04-23T01:51:11.631556+0000","last_clean":"2024-04-23T01:51:11.631556+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:51:11.631556+0000","last_undegraded":"2024-04-23T01:51:11.631556+0000","last_fullsized":"2024-04-23T01:51:11.631556+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"40'288","reported_seq":303,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-23T01:51:10.811853+0000","last_change":"2024-04-23T01:50:40.462462+0000","last_active":"2024-04-23T01:51:10.811853+0000","last_peered":"2024-04-23T01:51:10.811853+0000","last_clean":"2024-04-23T01:51:10.811853+0000","last_became_active":"2024-04-23T01:50:40.439584+0000","last_became_peered":"2024-04-23T01:50:40.439584+0000","last_unstale":"2024-04-23T01:51:10.811853+0000","last_undegraded":"2024-04-23T01:51:10.811853+0000","last_fullsized":"2024-04-23T01:51:10.811853+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":288,"log_dups_size":0,"ondisk_log_size":288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":83,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:10.817459+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:51:10.817459+0000","last_peered":"2024-04-23T01:51:10.817459+0000","last_clean":"2024-04-23T01:51:10.817459+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:51:10.817459+0000","last_undegraded":"2024-04-23T01:51:10.817459+0000","last_fullsized":"2024-04-23T01:51:10.817459+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"41'304","reported_seq":322,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.344169+0000","last_change":"2024-04-23T01:50:40.466468+0000","last_active":"2024-04-23T01:51:11.344169+0000","last_peered":"2024-04-23T01:51:11.344169+0000","last_clean":"2024-04-23T01:51:11.344169+0000","last_became_active":"2024-04-23T01:50:40.452408+0000","last_became_peered":"2024-04-23T01:50:40.452408+0000","last_unstale":"2024-04-23T01:51:11.344169+0000","last_undegraded":"2024-04-23T01:51:11.344169+0000","last_fullsized":"2024-04-23T01:51:11.344169+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490368,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":92,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.356370+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:51:11.356370+0000","last_peered":"2024-04-23T01:51:11.356370+0000","last_clean":"2024-04-23T01:51:11.356370+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:51:11.356370+0000","last_undegraded":"2024-04-23T01:51:11.356370+0000","last_fullsized":"2024-04-23T01:51:11.356370+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"40'360","reported_seq":378,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.350070+0000","last_change":"2024-04-23T01:50:40.469709+0000","last_active":"2024-04-23T01:51:11.350070+0000","last_peered":"2024-04-23T01:51:11.350070+0000","last_clean":"2024-04-23T01:51:11.350070+0000","last_became_active":"2024-04-23T01:50:40.462062+0000","last_became_peered":"2024-04-23T01:50:40.462062+0000","last_unstale":"2024-04-23T01:51:11.350070+0000","last_undegraded":"2024-04-23T01:51:11.350070+0000","last_fullsized":"2024-04-23T01:51:11.350070+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2949120,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":360,"num_write_kb":2880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":85,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-23T01:51:10.812233+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:51:10.812233+0000","last_peered":"2024-04-23T01:51:10.812233+0000","last_clean":"2024-04-23T01:51:10.812233+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:51:10.812233+0000","last_undegraded":"2024-04-23T01:51:10.812233+0000","last_fullsized":"2024-04-23T01:51:10.812233+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"41'312","reported_seq":330,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.621168+0000","last_change":"2024-04-23T01:50:40.464715+0000","last_active":"2024-04-23T01:51:11.621168+0000","last_peered":"2024-04-23T01:51:11.621168+0000","last_clean":"2024-04-23T01:51:11.621168+0000","last_became_active":"2024-04-23T01:50:40.457999+0000","last_became_peered":"2024-04-23T01:50:40.457999+0000","last_unstale":"2024-04-23T01:51:11.621168+0000","last_undegraded":"2024-04-23T01:51:11.621168+0000","last_fullsized":"2024-04-23T01:51:11.621168+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2555904,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":2496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":92,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.361042+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:51:11.361042+0000","last_peered":"2024-04-23T01:51:11.361042+0000","last_clean":"2024-04-23T01:51:11.361042+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:51:11.361042+0000","last_undegraded":"2024-04-23T01:51:11.361042+0000","last_fullsized":"2024-04-23T01:51:11.361042+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"40'344","reported_seq":359,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:10.802924+0000","last_change":"2024-04-23T01:50:40.447151+0000","last_active":"2024-04-23T01:51:10.802924+0000","last_peered":"2024-04-23T01:51:10.802924+0000","last_clean":"2024-04-23T01:51:10.802924+0000","last_became_active":"2024-04-23T01:50:40.441283+0000","last_became_peered":"2024-04-23T01:50:40.441283+0000","last_unstale":"2024-04-23T01:51:10.802924+0000","last_undegraded":"2024-04-23T01:51:10.802924+0000","last_fullsized":"2024-04-23T01:51:10.802924+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":344,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":88,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.611598+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:51:11.611598+0000","last_peered":"2024-04-23T01:51:11.611598+0000","last_clean":"2024-04-23T01:51:11.611598+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:51:11.611598+0000","last_undegraded":"2024-04-23T01:51:11.611598+0000","last_fullsized":"2024-04-23T01:51:11.611598+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":37330944,"num_objects":570,"num_object_clones":0,"num_object_copies":1710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4557,"num_write_kb":36456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":55992320,"data_stored":111984640,"data_compressed":5779080,"data_compressed_allocated":55992320,"data_compressed_original":111984640,"omap_allocated":0,"internal_metadata":0},"log_size":4553,"ondisk_log_size":4553,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":79892,"kb_used_data":12812,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94291948,"statfs":{"total":96636764160,"available":96554954752,"internally_reserved":0,"allocated":13119488,"data_stored":26124106,"data_compressed":1341297,"data_compressed_allocated":12746752,"data_compressed_original":25985024,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":88068,"kb_used_data":12668,"kb_used_omap":0,"kb_used_meta":75392,"kb_avail":94283772,"statfs":{"total":96636764160,"available":96546582528,"internally_reserved":0,"allocated":12972032,"data_stored":25829194,"data_compressed":1326129,"data_compressed_allocated":12599296,"data_compressed_original":25690112,"omap_allocated":0,"internal_metadata":77201408},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":90328,"kb_used_data":14928,"kb_used_omap":0,"kb_used_meta":75392,"kb_avail":94281512,"statfs":{"total":96636764160,"available":96544268288,"internally_reserved":0,"allocated":15286272,"data_stored":29990186,"data_compressed":1541334,"data_compressed_allocated":14934016,"data_compressed_original":29868032,"omap_allocated":0,"internal_metadata":77201408},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77816,"kb_used_data":14896,"kb_used_omap":0,"kb_used_meta":62912,"kb_avail":94294024,"statfs":{"total":96636764160,"available":96557080576,"internally_reserved":0,"allocated":15253504,"data_stored":30392138,"data_compressed":1561620,"data_compressed_allocated":14880768,"data_compressed_original":30253056,"omap_allocated":0,"internal_metadata":64421888},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15060992,"data_stored":30121984,"data_compressed":1554509,"data_compressed_allocated":15060992,"data_compressed_original":30121984,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15155200,"data_stored":30310400,"data_compressed":1564176,"data_compressed_allocated":15155200,"data_compressed_original":30310400,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12832768,"data_stored":25665536,"data_compressed":1324517,"data_compressed_allocated":12832768,"data_compressed_original":25665536,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12943360,"data_stored":25886720,"data_compressed":1335878,"data_compressed_allocated":12943360,"data_compressed_original":25886720,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:51:13.778 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:51:13.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:13.827+0000 7f040e154640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:13.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:13.834+0000 7f040e154640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:13.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:13.911+0000 7f4cfcead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:13.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:13.921+0000 7f4cfcead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:14.162 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:51:14.162 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:51:14.180 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-04-23T01:51:13.619383+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":37921312,"num_objects":572,"num_object_clones":0,"num_object_copies":1716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":4694,"num_write_kb":38350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4650,"ondisk_log_size":4650,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":336104,"kb_used_data":55304,"kb_used_omap":0,"kb_used_meta":280768,"kb_avail":377151256,"statfs":{"total":386547056640,"available":386202886144,"internally_reserved":0,"allocated":56631296,"data_stored":112335624,"data_compressed":5770380,"data_compressed_allocated":55160832,"data_compressed_original":111796224,"omap_allocated":0,"internal_metadata":287506432},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5808128,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":709,"num_write_kb":5672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000637"},"pg_stats":[{"pgid":"6.f","version":"42'284","reported_seq":303,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.606123+0000","last_change":"2024-04-23T01:50:40.440701+0000","last_active":"2024-04-23T01:51:11.606123+0000","last_peered":"2024-04-23T01:51:11.606123+0000","last_clean":"2024-04-23T01:51:11.606123+0000","last_became_active":"2024-04-23T01:50:40.433832+0000","last_became_peered":"2024-04-23T01:50:40.433832+0000","last_unstale":"2024-04-23T01:51:11.606123+0000","last_undegraded":"2024-04-23T01:51:11.606123+0000","last_fullsized":"2024-04-23T01:51:11.606123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2334720,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":285,"num_write_kb":2280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"41'256","reported_seq":274,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.635941+0000","last_change":"2024-04-23T01:50:40.477278+0000","last_active":"2024-04-23T01:51:11.635941+0000","last_peered":"2024-04-23T01:51:11.635941+0000","last_clean":"2024-04-23T01:51:11.635941+0000","last_became_active":"2024-04-23T01:50:40.471274+0000","last_became_peered":"2024-04-23T01:50:40.471274+0000","last_unstale":"2024-04-23T01:51:11.635941+0000","last_undegraded":"2024-04-23T01:51:11.635941+0000","last_fullsized":"2024-04-23T01:51:11.635941+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":32,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":256,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"41'304","reported_seq":322,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.624878+0000","last_change":"2024-04-23T01:50:40.446224+0000","last_active":"2024-04-23T01:51:11.624878+0000","last_peered":"2024-04-23T01:51:11.624878+0000","last_clean":"2024-04-23T01:51:11.624878+0000","last_became_active":"2024-04-23T01:50:40.437838+0000","last_became_peered":"2024-04-23T01:50:40.437838+0000","last_unstale":"2024-04-23T01:51:11.624878+0000","last_undegraded":"2024-04-23T01:51:11.624878+0000","last_fullsized":"2024-04-23T01:51:11.624878+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490368,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"40'208","reported_seq":223,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-23T01:51:10.818987+0000","last_change":"2024-04-23T01:50:40.517607+0000","last_active":"2024-04-23T01:51:10.818987+0000","last_peered":"2024-04-23T01:51:10.818987+0000","last_clean":"2024-04-23T01:51:10.818987+0000","last_became_active":"2024-04-23T01:50:40.511881+0000","last_became_peered":"2024-04-23T01:50:40.511881+0000","last_unstale":"2024-04-23T01:51:10.818987+0000","last_undegraded":"2024-04-23T01:51:10.818987+0000","last_fullsized":"2024-04-23T01:51:10.818987+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703936,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":208,"num_write_kb":1664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"41'352","reported_seq":370,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.630159+0000","last_change":"2024-04-23T01:50:40.432817+0000","last_active":"2024-04-23T01:51:11.630159+0000","last_peered":"2024-04-23T01:51:11.630159+0000","last_clean":"2024-04-23T01:51:11.630159+0000","last_became_active":"2024-04-23T01:50:40.421722+0000","last_became_peered":"2024-04-23T01:50:40.421722+0000","last_unstale":"2024-04-23T01:51:11.630159+0000","last_undegraded":"2024-04-23T01:51:11.630159+0000","last_fullsized":"2024-04-23T01:51:11.630159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883584,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":352,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"42'183","reported_seq":202,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.542547+0000","last_change":"2024-04-23T01:50:40.444510+0000","last_active":"2024-04-23T01:51:11.542547+0000","last_peered":"2024-04-23T01:51:11.542547+0000","last_clean":"2024-04-23T01:51:11.542547+0000","last_became_active":"2024-04-23T01:50:40.437088+0000","last_became_peered":"2024-04-23T01:50:40.437088+0000","last_unstale":"2024-04-23T01:51:11.542547+0000","last_undegraded":"2024-04-23T01:51:11.542547+0000","last_fullsized":"2024-04-23T01:51:11.542547+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"40'320","reported_seq":335,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-23T01:51:10.818793+0000","last_change":"2024-04-23T01:50:40.468123+0000","last_active":"2024-04-23T01:51:10.818793+0000","last_peered":"2024-04-23T01:51:10.818793+0000","last_clean":"2024-04-23T01:51:10.818793+0000","last_became_active":"2024-04-23T01:50:40.443838+0000","last_became_peered":"2024-04-23T01:50:40.443838+0000","last_unstale":"2024-04-23T01:51:10.818793+0000","last_undegraded":"2024-04-23T01:51:10.818793+0000","last_fullsized":"2024-04-23T01:51:10.818793+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":320,"num_write_kb":2560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"41'263","reported_seq":279,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.543637+0000","last_change":"2024-04-23T01:50:40.451743+0000","last_active":"2024-04-23T01:51:11.543637+0000","last_peered":"2024-04-23T01:51:11.543637+0000","last_clean":"2024-04-23T01:51:11.543637+0000","last_became_active":"2024-04-23T01:50:40.447092+0000","last_became_peered":"2024-04-23T01:50:40.447092+0000","last_unstale":"2024-04-23T01:51:11.543637+0000","last_undegraded":"2024-04-23T01:51:11.543637+0000","last_fullsized":"2024-04-23T01:51:11.543637+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162688,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":264,"num_write_kb":2112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"40'248","reported_seq":263,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:10.806777+0000","last_change":"2024-04-23T01:50:40.477557+0000","last_active":"2024-04-23T01:51:10.806777+0000","last_peered":"2024-04-23T01:51:10.806777+0000","last_clean":"2024-04-23T01:51:10.806777+0000","last_became_active":"2024-04-23T01:50:40.472479+0000","last_became_peered":"2024-04-23T01:50:40.472479+0000","last_unstale":"2024-04-23T01:51:10.806777+0000","last_undegraded":"2024-04-23T01:51:10.806777+0000","last_fullsized":"2024-04-23T01:51:10.806777+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":218,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-23T01:51:10.817606+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:51:10.817606+0000","last_peered":"2024-04-23T01:51:10.817606+0000","last_clean":"2024-04-23T01:51:10.817606+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:51:10.817606+0000","last_undegraded":"2024-04-23T01:51:10.817606+0000","last_fullsized":"2024-04-23T01:51:10.817606+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":88,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.628370+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:51:11.628370+0000","last_peered":"2024-04-23T01:51:11.628370+0000","last_clean":"2024-04-23T01:51:11.628370+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:51:11.628370+0000","last_undegraded":"2024-04-23T01:51:11.628370+0000","last_fullsized":"2024-04-23T01:51:11.628370+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"41'239","reported_seq":255,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.381514+0000","last_change":"2024-04-23T01:50:40.431365+0000","last_active":"2024-04-23T01:51:11.381514+0000","last_peered":"2024-04-23T01:51:11.381514+0000","last_clean":"2024-04-23T01:51:11.381514+0000","last_became_active":"2024-04-23T01:50:40.426237+0000","last_became_peered":"2024-04-23T01:50:40.426237+0000","last_unstale":"2024-04-23T01:51:11.381514+0000","last_undegraded":"2024-04-23T01:51:11.381514+0000","last_fullsized":"2024-04-23T01:51:11.381514+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":240,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":83,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:10.806611+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:51:10.806611+0000","last_peered":"2024-04-23T01:51:10.806611+0000","last_clean":"2024-04-23T01:51:10.806611+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:51:10.806611+0000","last_undegraded":"2024-04-23T01:51:10.806611+0000","last_fullsized":"2024-04-23T01:51:10.806611+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"41'288","reported_seq":306,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.341280+0000","last_change":"2024-04-23T01:50:40.486856+0000","last_active":"2024-04-23T01:51:11.341280+0000","last_peered":"2024-04-23T01:51:11.341280+0000","last_clean":"2024-04-23T01:51:11.341280+0000","last_became_active":"2024-04-23T01:50:40.478453+0000","last_became_peered":"2024-04-23T01:50:40.478453+0000","last_unstale":"2024-04-23T01:51:11.341280+0000","last_undegraded":"2024-04-23T01:51:11.341280+0000","last_fullsized":"2024-04-23T01:51:11.341280+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":288,"log_dups_size":0,"ondisk_log_size":288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":88,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.631556+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:51:11.631556+0000","last_peered":"2024-04-23T01:51:11.631556+0000","last_clean":"2024-04-23T01:51:11.631556+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:51:11.631556+0000","last_undegraded":"2024-04-23T01:51:11.631556+0000","last_fullsized":"2024-04-23T01:51:11.631556+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"40'288","reported_seq":303,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-23T01:51:10.811853+0000","last_change":"2024-04-23T01:50:40.462462+0000","last_active":"2024-04-23T01:51:10.811853+0000","last_peered":"2024-04-23T01:51:10.811853+0000","last_clean":"2024-04-23T01:51:10.811853+0000","last_became_active":"2024-04-23T01:50:40.439584+0000","last_became_peered":"2024-04-23T01:50:40.439584+0000","last_unstale":"2024-04-23T01:51:10.811853+0000","last_undegraded":"2024-04-23T01:51:10.811853+0000","last_fullsized":"2024-04-23T01:51:10.811853+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":288,"log_dups_size":0,"ondisk_log_size":288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":83,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:10.817459+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:51:10.817459+0000","last_peered":"2024-04-23T01:51:10.817459+0000","last_clean":"2024-04-23T01:51:10.817459+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:51:10.817459+0000","last_undegraded":"2024-04-23T01:51:10.817459+0000","last_fullsized":"2024-04-23T01:51:10.817459+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"41'304","reported_seq":322,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.344169+0000","last_change":"2024-04-23T01:50:40.466468+0000","last_active":"2024-04-23T01:51:11.344169+0000","last_peered":"2024-04-23T01:51:11.344169+0000","last_clean":"2024-04-23T01:51:11.344169+0000","last_became_active":"2024-04-23T01:50:40.452408+0000","last_became_peered":"2024-04-23T01:50:40.452408+0000","last_unstale":"2024-04-23T01:51:11.344169+0000","last_undegraded":"2024-04-23T01:51:11.344169+0000","last_fullsized":"2024-04-23T01:51:11.344169+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490368,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":92,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.356370+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:51:11.356370+0000","last_peered":"2024-04-23T01:51:11.356370+0000","last_clean":"2024-04-23T01:51:11.356370+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:51:11.356370+0000","last_undegraded":"2024-04-23T01:51:11.356370+0000","last_fullsized":"2024-04-23T01:51:11.356370+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"40'360","reported_seq":378,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.350070+0000","last_change":"2024-04-23T01:50:40.469709+0000","last_active":"2024-04-23T01:51:11.350070+0000","last_peered":"2024-04-23T01:51:11.350070+0000","last_clean":"2024-04-23T01:51:11.350070+0000","last_became_active":"2024-04-23T01:50:40.462062+0000","last_became_peered":"2024-04-23T01:50:40.462062+0000","last_unstale":"2024-04-23T01:51:11.350070+0000","last_undegraded":"2024-04-23T01:51:11.350070+0000","last_fullsized":"2024-04-23T01:51:11.350070+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2949120,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":360,"num_write_kb":2880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":85,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-23T01:51:10.812233+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:51:10.812233+0000","last_peered":"2024-04-23T01:51:10.812233+0000","last_clean":"2024-04-23T01:51:10.812233+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:51:10.812233+0000","last_undegraded":"2024-04-23T01:51:10.812233+0000","last_fullsized":"2024-04-23T01:51:10.812233+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"41'312","reported_seq":330,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.621168+0000","last_change":"2024-04-23T01:50:40.464715+0000","last_active":"2024-04-23T01:51:11.621168+0000","last_peered":"2024-04-23T01:51:11.621168+0000","last_clean":"2024-04-23T01:51:11.621168+0000","last_became_active":"2024-04-23T01:50:40.457999+0000","last_became_peered":"2024-04-23T01:50:40.457999+0000","last_unstale":"2024-04-23T01:51:11.621168+0000","last_undegraded":"2024-04-23T01:51:11.621168+0000","last_fullsized":"2024-04-23T01:51:11.621168+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2555904,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":2496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":92,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.361042+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:51:11.361042+0000","last_peered":"2024-04-23T01:51:11.361042+0000","last_clean":"2024-04-23T01:51:11.361042+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:51:11.361042+0000","last_undegraded":"2024-04-23T01:51:11.361042+0000","last_fullsized":"2024-04-23T01:51:11.361042+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"40'344","reported_seq":359,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:10.802924+0000","last_change":"2024-04-23T01:50:40.447151+0000","last_active":"2024-04-23T01:51:10.802924+0000","last_peered":"2024-04-23T01:51:10.802924+0000","last_clean":"2024-04-23T01:51:10.802924+0000","last_became_active":"2024-04-23T01:50:40.441283+0000","last_became_peered":"2024-04-23T01:50:40.441283+0000","last_unstale":"2024-04-23T01:51:10.802924+0000","last_undegraded":"2024-04-23T01:51:10.802924+0000","last_fullsized":"2024-04-23T01:51:10.802924+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":344,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":88,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-23T01:51:11.611598+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:51:11.611598+0000","last_peered":"2024-04-23T01:51:11.611598+0000","last_clean":"2024-04-23T01:51:11.611598+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:51:11.611598+0000","last_undegraded":"2024-04-23T01:51:11.611598+0000","last_fullsized":"2024-04-23T01:51:11.611598+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":37330944,"num_objects":570,"num_object_clones":0,"num_object_copies":1710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4557,"num_write_kb":36456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":55992320,"data_stored":111984640,"data_compressed":5779080,"data_compressed_allocated":55992320,"data_compressed_original":111984640,"omap_allocated":0,"internal_metadata":0},"log_size":4553,"ondisk_log_size":4553,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":79892,"kb_used_data":12812,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94291948,"statfs":{"total":96636764160,"available":96554954752,"internally_reserved":0,"allocated":13119488,"data_stored":26124106,"data_compressed":1341297,"data_compressed_allocated":12746752,"data_compressed_original":25985024,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":88068,"kb_used_data":12668,"kb_used_omap":0,"kb_used_meta":75392,"kb_avail":94283772,"statfs":{"total":96636764160,"available":96546582528,"internally_reserved":0,"allocated":12972032,"data_stored":25829194,"data_compressed":1326129,"data_compressed_allocated":12599296,"data_compressed_original":25690112,"omap_allocated":0,"internal_metadata":77201408},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":90328,"kb_used_data":14928,"kb_used_omap":0,"kb_used_meta":75392,"kb_avail":94281512,"statfs":{"total":96636764160,"available":96544268288,"internally_reserved":0,"allocated":15286272,"data_stored":29990186,"data_compressed":1541334,"data_compressed_allocated":14934016,"data_compressed_original":29868032,"omap_allocated":0,"internal_metadata":77201408},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77816,"kb_used_data":14896,"kb_used_omap":0,"kb_used_meta":62912,"kb_avail":94294024,"statfs":{"total":96636764160,"available":96557080576,"internally_reserved":0,"allocated":15253504,"data_stored":30392138,"data_compressed":1561620,"data_compressed_allocated":14880768,"data_compressed_original":30253056,"omap_allocated":0,"internal_metadata":64421888},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15060992,"data_stored":30121984,"data_compressed":1554509,"data_compressed_allocated":15060992,"data_compressed_original":30121984,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15155200,"data_stored":30310400,"data_compressed":1564176,"data_compressed_allocated":15155200,"data_compressed_original":30310400,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12832768,"data_stored":25665536,"data_compressed":1324517,"data_compressed_allocated":12832768,"data_compressed_original":25665536,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12943360,"data_stored":25886720,"data_compressed":1335878,"data_compressed_allocated":12943360,"data_compressed_original":25886720,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:51:14.182 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-23T01:51:14.182 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:51:14.182 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:51:14.182 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:51:14.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:14.309+0000 7f990baed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:14.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:14.322+0000 7f990baed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:14.347 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:51:14.347 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:51:14.347 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:51:14.347 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:51:14.347 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:51:14.347 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:51:14.347 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:51:14.347 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:51:14.348 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:51:14.348 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:51:14.348 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:51:14.348 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:51:14.351 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:51:14.351 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:51:14.351 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:51:14.354 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 33 16 4957 4941 1.16964 1.09375 0.129653 0.106648 2024-04-23T01:51:14.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:14.504+0000 7fb37b86d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:14.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:14.517+0000 7fb37b86d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:14.591 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:51:14.591 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:51:14.591 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:51:14.591 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:51:14.592 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:51:14.592 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:51:14.592 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:51:14.592 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:51:14.592 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:51:14.592 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:51:14.592 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:51:14.592 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:51:14.594 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:51:14.594 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:51:14.595 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:51:14.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:14.717+0000 7f293282d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:14.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:14.729+0000 7f293282d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:14.750 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:51:14.750 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:51:14.750 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:51:14.750 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:51:14.750 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:51:14.750 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:51:14.750 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:51:14.751 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:51:14.751 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:51:14.751 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:51:14.751 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:51:14.751 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:51:14.754 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:51:14.754 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:51:14.755 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:51:14.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:14.890+0000 7f914419d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:14.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:14.901+0000 7f914419d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:14.924 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:51:14.924 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:51:14.925 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:51:14.925 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:51:14.925 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:51:14.925 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:51:14.925 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:51:14.925 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:51:14.925 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:51:14.925 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:51:14.925 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:51:14.925 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:51:14.928 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:51:14.928 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:51:15.353 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 34 16 5114 5098 1.17132 1.22656 0.0687532 0.106572 2024-04-23T01:51:16.043 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:51:16.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:16.164+0000 7f2a3991d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:16.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:16.171+0000 7f2a3991d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:16.353 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 35 16 5251 5235 1.16843 1.07031 0.0248891 0.106676 2024-04-23T01:51:17.354 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 36 16 5383 5367 1.16461 1.03125 0.112086 0.107112 2024-04-23T01:51:18.315 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:51:18.353 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 37 16 5523 5507 1.1627 1.09375 0.108611 0.107306 2024-04-23T01:51:18.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:18.442+0000 7f5ceee17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:18.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:18.448+0000 7f5ceee17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:19.354 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 38 16 5682 5666 1.16479 1.24219 0.0975871 0.10715 2024-04-23T01:51:19.929 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:51:19.929 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:51:19.930 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:51:20.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:20.060+0000 7f60ca82d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:20.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:20.071+0000 7f60ca82d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:20.354 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 39 16 5824 5808 1.16336 1.10938 0.087685 0.107096 2024-04-23T01:51:20.426 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:51:20.427 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:51:20.447 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":224,"stamp":"2024-04-23T01:51:19.620382+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":43860512,"num_objects":664,"num_object_clones":0,"num_object_copies":1992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":5419,"num_write_kb":44150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5364,"ondisk_log_size":5364,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":340668,"kb_used_data":64028,"kb_used_omap":0,"kb_used_meta":276608,"kb_avail":377146692,"statfs":{"total":386547056640,"available":386198212608,"internally_reserved":0,"allocated":65564672,"data_stored":130084125,"data_compressed":6685329,"data_compressed_allocated":64020480,"data_compressed_original":129515520,"omap_allocated":0,"internal_metadata":283246592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11747328,"num_objects":179,"num_object_clones":0,"num_object_copies":537,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1434,"num_write_kb":11472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001306"},"pg_stats":[{"pgid":"6.f","version":"43'335","reported_seq":357,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T01:51:16.432737+0000","last_change":"2024-04-23T01:50:40.440701+0000","last_active":"2024-04-23T01:51:16.432737+0000","last_peered":"2024-04-23T01:51:16.432737+0000","last_clean":"2024-04-23T01:51:16.432737+0000","last_became_active":"2024-04-23T01:50:40.433832+0000","last_became_peered":"2024-04-23T01:50:40.433832+0000","last_unstale":"2024-04-23T01:51:16.432737+0000","last_undegraded":"2024-04-23T01:51:16.432737+0000","last_fullsized":"2024-04-23T01:51:16.432737+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"43'295","reported_seq":317,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T01:51:16.268979+0000","last_change":"2024-04-23T01:50:40.477278+0000","last_active":"2024-04-23T01:51:16.268979+0000","last_peered":"2024-04-23T01:51:16.268979+0000","last_clean":"2024-04-23T01:51:16.268979+0000","last_became_active":"2024-04-23T01:50:40.471274+0000","last_became_peered":"2024-04-23T01:50:40.471274+0000","last_unstale":"2024-04-23T01:51:16.268979+0000","last_undegraded":"2024-04-23T01:51:16.268979+0000","last_fullsized":"2024-04-23T01:51:16.268979+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2424832,"num_objects":37,"num_object_clones":0,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":296,"num_write_kb":2368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"43'335","reported_seq":357,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T01:51:14.822738+0000","last_change":"2024-04-23T01:50:40.446224+0000","last_active":"2024-04-23T01:51:14.822738+0000","last_peered":"2024-04-23T01:51:14.822738+0000","last_clean":"2024-04-23T01:51:14.822738+0000","last_became_active":"2024-04-23T01:50:40.437838+0000","last_became_peered":"2024-04-23T01:50:40.437838+0000","last_unstale":"2024-04-23T01:51:14.822738+0000","last_undegraded":"2024-04-23T01:51:14.822738+0000","last_fullsized":"2024-04-23T01:51:14.822738+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"40'208","reported_seq":232,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:13.381690+0000","last_change":"2024-04-23T01:50:40.517607+0000","last_active":"2024-04-23T01:51:13.381690+0000","last_peered":"2024-04-23T01:51:13.381690+0000","last_clean":"2024-04-23T01:51:13.381690+0000","last_became_active":"2024-04-23T01:50:40.511881+0000","last_became_peered":"2024-04-23T01:50:40.511881+0000","last_unstale":"2024-04-23T01:51:13.381690+0000","last_undegraded":"2024-04-23T01:51:13.381690+0000","last_fullsized":"2024-04-23T01:51:13.381690+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703936,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":208,"num_write_kb":1664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"43'399","reported_seq":421,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T01:51:16.155746+0000","last_change":"2024-04-23T01:50:40.432817+0000","last_active":"2024-04-23T01:51:16.155746+0000","last_peered":"2024-04-23T01:51:16.155746+0000","last_clean":"2024-04-23T01:51:16.155746+0000","last_became_active":"2024-04-23T01:50:40.421722+0000","last_became_peered":"2024-04-23T01:50:40.421722+0000","last_unstale":"2024-04-23T01:51:16.155746+0000","last_undegraded":"2024-04-23T01:51:16.155746+0000","last_fullsized":"2024-04-23T01:51:16.155746+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3276800,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":400,"num_write_kb":3200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"43'239","reported_seq":261,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T01:51:15.300421+0000","last_change":"2024-04-23T01:50:40.444510+0000","last_active":"2024-04-23T01:51:15.300421+0000","last_peered":"2024-04-23T01:51:15.300421+0000","last_clean":"2024-04-23T01:51:15.300421+0000","last_became_active":"2024-04-23T01:50:40.437088+0000","last_became_peered":"2024-04-23T01:50:40.437088+0000","last_unstale":"2024-04-23T01:51:15.300421+0000","last_undegraded":"2024-04-23T01:51:15.300421+0000","last_fullsized":"2024-04-23T01:51:15.300421+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":240,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"44'367","reported_seq":392,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:16.557561+0000","last_change":"2024-04-23T01:50:40.468123+0000","last_active":"2024-04-23T01:51:16.557561+0000","last_peered":"2024-04-23T01:51:16.557561+0000","last_clean":"2024-04-23T01:51:16.557561+0000","last_became_active":"2024-04-23T01:50:40.443838+0000","last_became_peered":"2024-04-23T01:50:40.443838+0000","last_unstale":"2024-04-23T01:51:16.557561+0000","last_undegraded":"2024-04-23T01:51:16.557561+0000","last_fullsized":"2024-04-23T01:51:16.557561+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3014656,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":368,"num_write_kb":2944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"43'327","reported_seq":349,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T01:51:15.626335+0000","last_change":"2024-04-23T01:50:40.451743+0000","last_active":"2024-04-23T01:51:15.626335+0000","last_peered":"2024-04-23T01:51:15.626335+0000","last_clean":"2024-04-23T01:51:15.626335+0000","last_became_active":"2024-04-23T01:50:40.447092+0000","last_became_peered":"2024-04-23T01:50:40.447092+0000","last_unstale":"2024-04-23T01:51:15.626335+0000","last_undegraded":"2024-04-23T01:51:15.626335+0000","last_fullsized":"2024-04-23T01:51:15.626335+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2686976,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":328,"num_write_kb":2624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"43'271","reported_seq":293,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T01:51:15.291448+0000","last_change":"2024-04-23T01:50:40.477557+0000","last_active":"2024-04-23T01:51:15.291448+0000","last_peered":"2024-04-23T01:51:15.291448+0000","last_clean":"2024-04-23T01:51:15.291448+0000","last_became_active":"2024-04-23T01:50:40.472479+0000","last_became_peered":"2024-04-23T01:50:40.472479+0000","last_unstale":"2024-04-23T01:51:15.291448+0000","last_undegraded":"2024-04-23T01:51:15.291448+0000","last_fullsized":"2024-04-23T01:51:15.291448+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":272,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":227,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:13.373808+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:51:13.373808+0000","last_peered":"2024-04-23T01:51:13.373808+0000","last_clean":"2024-04-23T01:51:13.373808+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:51:13.373808+0000","last_undegraded":"2024-04-23T01:51:13.373808+0000","last_fullsized":"2024-04-23T01:51:13.373808+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":91,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T01:51:12.378645+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:51:12.378645+0000","last_peered":"2024-04-23T01:51:12.378645+0000","last_clean":"2024-04-23T01:51:12.378645+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:51:12.378645+0000","last_undegraded":"2024-04-23T01:51:12.378645+0000","last_fullsized":"2024-04-23T01:51:12.378645+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"44'309","reported_seq":334,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:16.577571+0000","last_change":"2024-04-23T01:50:40.431365+0000","last_active":"2024-04-23T01:51:16.577571+0000","last_peered":"2024-04-23T01:51:16.577571+0000","last_clean":"2024-04-23T01:51:16.577571+0000","last_became_active":"2024-04-23T01:50:40.426237+0000","last_became_peered":"2024-04-23T01:50:40.426237+0000","last_unstale":"2024-04-23T01:51:16.577571+0000","last_undegraded":"2024-04-23T01:51:16.577571+0000","last_fullsized":"2024-04-23T01:51:16.577571+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2539520,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":310,"num_write_kb":2480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T01:51:12.475122+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:51:12.475122+0000","last_peered":"2024-04-23T01:51:12.475122+0000","last_clean":"2024-04-23T01:51:12.475122+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:51:12.475122+0000","last_undegraded":"2024-04-23T01:51:12.475122+0000","last_fullsized":"2024-04-23T01:51:12.475122+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"43'344","reported_seq":366,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T01:51:15.944008+0000","last_change":"2024-04-23T01:50:40.486856+0000","last_active":"2024-04-23T01:51:15.944008+0000","last_peered":"2024-04-23T01:51:15.944008+0000","last_clean":"2024-04-23T01:51:15.944008+0000","last_became_active":"2024-04-23T01:50:40.478453+0000","last_became_peered":"2024-04-23T01:50:40.478453+0000","last_unstale":"2024-04-23T01:51:15.944008+0000","last_undegraded":"2024-04-23T01:51:15.944008+0000","last_fullsized":"2024-04-23T01:51:15.944008+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2826240,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":345,"num_write_kb":2760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":91,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T01:51:12.364696+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:51:12.364696+0000","last_peered":"2024-04-23T01:51:12.364696+0000","last_clean":"2024-04-23T01:51:12.364696+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:51:12.364696+0000","last_undegraded":"2024-04-23T01:51:12.364696+0000","last_fullsized":"2024-04-23T01:51:12.364696+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"44'311","reported_seq":336,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:15.834061+0000","last_change":"2024-04-23T01:50:40.462462+0000","last_active":"2024-04-23T01:51:15.834061+0000","last_peered":"2024-04-23T01:51:15.834061+0000","last_clean":"2024-04-23T01:51:15.834061+0000","last_became_active":"2024-04-23T01:50:40.439584+0000","last_became_peered":"2024-04-23T01:50:40.439584+0000","last_unstale":"2024-04-23T01:51:15.834061+0000","last_undegraded":"2024-04-23T01:51:15.834061+0000","last_fullsized":"2024-04-23T01:51:15.834061+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2555904,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":2496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T01:51:12.475669+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:51:12.475669+0000","last_peered":"2024-04-23T01:51:12.475669+0000","last_clean":"2024-04-23T01:51:12.475669+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:51:12.475669+0000","last_undegraded":"2024-04-23T01:51:12.475669+0000","last_fullsized":"2024-04-23T01:51:12.475669+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"43'383","reported_seq":405,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T01:51:16.494662+0000","last_change":"2024-04-23T01:50:40.466468+0000","last_active":"2024-04-23T01:51:16.494662+0000","last_peered":"2024-04-23T01:51:16.494662+0000","last_clean":"2024-04-23T01:51:16.494662+0000","last_became_active":"2024-04-23T01:50:40.452408+0000","last_became_peered":"2024-04-23T01:50:40.452408+0000","last_unstale":"2024-04-23T01:51:16.494662+0000","last_undegraded":"2024-04-23T01:51:16.494662+0000","last_fullsized":"2024-04-23T01:51:16.494662+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":384,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T01:51:12.494435+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:51:12.494435+0000","last_peered":"2024-04-23T01:51:12.494435+0000","last_clean":"2024-04-23T01:51:12.494435+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:51:12.494435+0000","last_undegraded":"2024-04-23T01:51:12.494435+0000","last_fullsized":"2024-04-23T01:51:12.494435+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"43'415","reported_seq":437,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T01:51:14.660690+0000","last_change":"2024-04-23T01:50:40.469709+0000","last_active":"2024-04-23T01:51:14.660690+0000","last_peered":"2024-04-23T01:51:14.660690+0000","last_clean":"2024-04-23T01:51:14.660690+0000","last_became_active":"2024-04-23T01:50:40.462062+0000","last_became_peered":"2024-04-23T01:50:40.462062+0000","last_unstale":"2024-04-23T01:51:14.660690+0000","last_undegraded":"2024-04-23T01:51:14.660690+0000","last_fullsized":"2024-04-23T01:51:14.660690+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3407872,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":3328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":94,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:13.380426+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:51:13.380426+0000","last_peered":"2024-04-23T01:51:13.380426+0000","last_clean":"2024-04-23T01:51:13.380426+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:51:13.380426+0000","last_undegraded":"2024-04-23T01:51:13.380426+0000","last_fullsized":"2024-04-23T01:51:13.380426+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"43'342","reported_seq":364,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T01:51:16.615890+0000","last_change":"2024-04-23T01:50:40.464715+0000","last_active":"2024-04-23T01:51:16.615890+0000","last_peered":"2024-04-23T01:51:16.615890+0000","last_clean":"2024-04-23T01:51:16.615890+0000","last_became_active":"2024-04-23T01:50:40.457999+0000","last_became_peered":"2024-04-23T01:50:40.457999+0000","last_unstale":"2024-04-23T01:51:16.615890+0000","last_undegraded":"2024-04-23T01:51:16.615890+0000","last_fullsized":"2024-04-23T01:51:16.615890+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2809856,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":343,"num_write_kb":2744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T01:51:12.488979+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:51:12.488979+0000","last_peered":"2024-04-23T01:51:12.488979+0000","last_clean":"2024-04-23T01:51:12.488979+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:51:12.488979+0000","last_undegraded":"2024-04-23T01:51:12.488979+0000","last_fullsized":"2024-04-23T01:51:12.488979+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"43'387","reported_seq":409,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T01:51:16.609991+0000","last_change":"2024-04-23T01:50:40.447151+0000","last_active":"2024-04-23T01:51:16.609991+0000","last_peered":"2024-04-23T01:51:16.609991+0000","last_clean":"2024-04-23T01:51:16.609991+0000","last_became_active":"2024-04-23T01:50:40.441283+0000","last_became_peered":"2024-04-23T01:50:40.441283+0000","last_unstale":"2024-04-23T01:51:16.609991+0000","last_undegraded":"2024-04-23T01:51:16.609991+0000","last_fullsized":"2024-04-23T01:51:16.609991+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3178496,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":388,"num_write_kb":3104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":91,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T01:51:12.355419+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:51:12.355419+0000","last_peered":"2024-04-23T01:51:12.355419+0000","last_clean":"2024-04-23T01:51:12.355419+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:51:12.355419+0000","last_undegraded":"2024-04-23T01:51:12.355419+0000","last_fullsized":"2024-04-23T01:51:12.355419+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":43270144,"num_objects":662,"num_object_clones":0,"num_object_copies":1986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5282,"num_write_kb":42256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":64897024,"data_stored":129794048,"data_compressed":6698682,"data_compressed_allocated":64897024,"data_compressed_original":129794048,"omap_allocated":0,"internal_metadata":0},"log_size":5267,"ondisk_log_size":5267,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706033,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":86024,"kb_used_data":14784,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94285816,"statfs":{"total":96636764160,"available":96548675584,"internally_reserved":0,"allocated":15138816,"data_stored":30123345,"data_compressed":1547298,"data_compressed_allocated":14741504,"data_compressed_original":29974528,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706033,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":90220,"kb_used_data":14820,"kb_used_omap":0,"kb_used_meta":75392,"kb_avail":94281620,"statfs":{"total":96636764160,"available":96544378880,"internally_reserved":0,"allocated":15175680,"data_stored":30210212,"data_compressed":1552011,"data_compressed_allocated":14786560,"data_compressed_original":30064640,"omap_allocated":0,"internal_metadata":77201408},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706033,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":79896,"kb_used_data":16976,"kb_used_omap":0,"kb_used_meta":62912,"kb_avail":94291944,"statfs":{"total":96636764160,"available":96554950656,"internally_reserved":0,"allocated":17383424,"data_stored":34158212,"data_compressed":1756218,"data_compressed_allocated":17014784,"data_compressed_original":34029568,"omap_allocated":0,"internal_metadata":64421888},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706033,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":84528,"kb_used_data":17448,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94287312,"statfs":{"total":96636764160,"available":96550207488,"internally_reserved":0,"allocated":17866752,"data_stored":35592356,"data_compressed":1829802,"data_compressed_allocated":17477632,"data_compressed_original":35446784,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17633280,"data_stored":35266560,"data_compressed":1820153,"data_compressed_allocated":17633280,"data_compressed_original":35266560,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":17305600,"data_stored":34611200,"data_compressed":1786251,"data_compressed_allocated":17305600,"data_compressed_original":34611200,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":15003648,"data_stored":30007296,"data_compressed":1548707,"data_compressed_allocated":15003648,"data_compressed_original":30007296,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":14954496,"data_stored":29908992,"data_compressed":1543571,"data_compressed_allocated":14954496,"data_compressed_original":29908992,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:51:20.650 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:51:20.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:20.785+0000 7f15b5d5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:20.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:20.793+0000 7f15b5d5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:21.354 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:51:21.354774+0000 min lat: 0.0144892 max lat: 0.333329 avg lat: 0.107551 2024-04-23T01:51:21.354 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:51:21.354 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 40 16 5958 5942 1.16045 1.04688 0.0840461 0.107551 2024-04-23T01:51:22.354 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 41 16 6108 6092 1.16072 1.17188 0.136917 0.107544 2024-04-23T01:51:22.919 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:51:23.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:23.059+0000 7f7b2f35d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:23.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:23.067+0000 7f7b2f35d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:23.354 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 42 16 6241 6225 1.15782 1.03906 0.110082 0.107729 2024-04-23T01:51:24.354 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 43 16 6382 6366 1.15651 1.10156 0.134764 0.107904 2024-04-23T01:51:25.310 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:51:25.354 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 44 16 6539 6523 1.1581 1.22656 0.0513604 0.107749 2024-04-23T01:51:25.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:25.434+0000 7f392199d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:25.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:25.441+0000 7f392199d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:25.449 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:51:25.449 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:51:25.449 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:51:25.449 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:51:25.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:25.574+0000 7fc6f52ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:25.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:25.588+0000 7fc6f52ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:25.629 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:51:25.629 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:51:25.630 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:51:25.630 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:51:25.630 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:51:25.630 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:51:25.630 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:51:25.630 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:51:25.630 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:51:25.630 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:51:25.630 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:51:25.630 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:51:25.632 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:51:25.633 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:51:25.633 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:51:25.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:25.805+0000 7ff52846d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:25.921 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:51:25.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:25.818+0000 7ff52846d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:25.921 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:51:25.921 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:51:25.921 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:51:25.921 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:51:25.921 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:51:25.921 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:51:25.921 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:51:25.922 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:51:25.922 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:51:25.922 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:51:25.922 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:51:25.922 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:51:25.922 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:51:25.922 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:51:26.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:26.079+0000 7f23a022d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:26.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:26.091+0000 7f23a022d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:26.109 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:51:26.109 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:51:26.109 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:51:26.109 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:51:26.109 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:51:26.109 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:51:26.109 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:51:26.109 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:51:26.109 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:51:26.109 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:51:26.110 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:51:26.110 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:51:26.113 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:51:26.113 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:51:26.114 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:51:26.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:26.243+0000 7ffbcad9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:26.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:26.255+0000 7ffbcad9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:26.276 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:51:26.276 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:51:26.276 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:51:26.277 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:51:26.277 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:51:26.277 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:51:26.277 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:51:26.277 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:51:26.277 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:51:26.277 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:51:26.277 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:51:26.277 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:51:26.279 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:51:26.279 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:51:26.354 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 45 16 6677 6661 1.15632 1.07812 0.0830715 0.10793 2024-04-23T01:51:27.354 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 46 16 6820 6804 1.15547 1.11719 0.129562 0.10804 2024-04-23T01:51:27.588 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:51:27.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:27.722+0000 7f923b8ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:27.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:27.729+0000 7f923b8ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:28.355 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 47 16 6971 6955 1.15598 1.17969 0.126687 0.107986 2024-04-23T01:51:29.355 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 48 16 7109 7093 1.15436 1.07812 0.0840297 0.108131 2024-04-23T01:51:29.980 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:51:30.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:30.095+0000 7f3acd59d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:30.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:30.101+0000 7f3acd59d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:30.355 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 49 16 7242 7226 1.15201 1.03906 0.0776887 0.108379 2024-04-23T01:51:31.280 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:51:31.280 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:51:31.280 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:51:31.355 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 50 16 7391 7375 1.15224 1.16406 0.198602 0.10836 2024-04-23T01:51:31.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:31.405+0000 7f936442d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:31.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:31.415+0000 7f936442d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:31.649 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:51:31.649 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:51:31.664 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-04-23T01:51:31.622566+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":55525920,"num_objects":842,"num_object_clones":0,"num_object_copies":2526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":6843,"num_write_kb":55542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6787,"ondisk_log_size":6787,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":357812,"kb_used_data":81172,"kb_used_omap":0,"kb_used_meta":276608,"kb_avail":377129548,"statfs":{"total":386547056640,"available":386180657152,"internally_reserved":0,"allocated":83120128,"data_stored":165155620,"data_compressed":8495769,"data_compressed_allocated":81551360,"data_compressed_original":164577280,"omap_allocated":0,"internal_metadata":283246592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11665408,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1424,"num_write_kb":11392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002183"},"pg_stats":[{"pgid":"6.f","version":"44'431","reported_seq":456,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:25.181890+0000","last_change":"2024-04-23T01:50:40.440701+0000","last_active":"2024-04-23T01:51:25.181890+0000","last_peered":"2024-04-23T01:51:25.181890+0000","last_clean":"2024-04-23T01:51:25.181890+0000","last_became_active":"2024-04-23T01:50:40.433832+0000","last_became_peered":"2024-04-23T01:50:40.433832+0000","last_unstale":"2024-04-23T01:51:25.181890+0000","last_undegraded":"2024-04-23T01:51:25.181890+0000","last_fullsized":"2024-04-23T01:51:25.181890+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3538944,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":432,"num_write_kb":3456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"44'405","reported_seq":430,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:26.603964+0000","last_change":"2024-04-23T01:50:40.477278+0000","last_active":"2024-04-23T01:51:26.603964+0000","last_peered":"2024-04-23T01:51:26.603964+0000","last_clean":"2024-04-23T01:51:26.603964+0000","last_became_active":"2024-04-23T01:50:40.471274+0000","last_became_peered":"2024-04-23T01:50:40.471274+0000","last_unstale":"2024-04-23T01:51:26.603964+0000","last_undegraded":"2024-04-23T01:51:26.603964+0000","last_fullsized":"2024-04-23T01:51:26.603964+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3325952,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":406,"num_write_kb":3248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"44'447","reported_seq":472,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:26.488161+0000","last_change":"2024-04-23T01:50:40.446224+0000","last_active":"2024-04-23T01:51:26.488161+0000","last_peered":"2024-04-23T01:51:26.488161+0000","last_clean":"2024-04-23T01:51:26.488161+0000","last_became_active":"2024-04-23T01:50:40.437838+0000","last_became_peered":"2024-04-23T01:50:40.437838+0000","last_unstale":"2024-04-23T01:51:26.488161+0000","last_undegraded":"2024-04-23T01:51:26.488161+0000","last_fullsized":"2024-04-23T01:51:26.488161+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":448,"num_write_kb":3584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"44'335","reported_seq":360,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:26.089626+0000","last_change":"2024-04-23T01:50:40.517607+0000","last_active":"2024-04-23T01:51:26.089626+0000","last_peered":"2024-04-23T01:51:26.089626+0000","last_clean":"2024-04-23T01:51:26.089626+0000","last_became_active":"2024-04-23T01:50:40.511881+0000","last_became_peered":"2024-04-23T01:50:40.511881+0000","last_unstale":"2024-04-23T01:51:26.089626+0000","last_undegraded":"2024-04-23T01:51:26.089626+0000","last_fullsized":"2024-04-23T01:51:26.089626+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'519","reported_seq":544,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:26.612924+0000","last_change":"2024-04-23T01:50:40.432817+0000","last_active":"2024-04-23T01:51:26.612924+0000","last_peered":"2024-04-23T01:51:26.612924+0000","last_clean":"2024-04-23T01:51:26.612924+0000","last_became_active":"2024-04-23T01:50:40.421722+0000","last_became_peered":"2024-04-23T01:50:40.421722+0000","last_unstale":"2024-04-23T01:51:26.612924+0000","last_undegraded":"2024-04-23T01:51:26.612924+0000","last_fullsized":"2024-04-23T01:51:26.612924+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259840,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":520,"num_write_kb":4160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"44'287","reported_seq":312,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:25.513477+0000","last_change":"2024-04-23T01:50:40.444510+0000","last_active":"2024-04-23T01:51:25.513477+0000","last_peered":"2024-04-23T01:51:25.513477+0000","last_clean":"2024-04-23T01:51:25.513477+0000","last_became_active":"2024-04-23T01:50:40.437088+0000","last_became_peered":"2024-04-23T01:50:40.437088+0000","last_unstale":"2024-04-23T01:51:25.513477+0000","last_undegraded":"2024-04-23T01:51:25.513477+0000","last_fullsized":"2024-04-23T01:51:25.513477+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"44'455","reported_seq":480,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:25.619851+0000","last_change":"2024-04-23T01:50:40.468123+0000","last_active":"2024-04-23T01:51:25.619851+0000","last_peered":"2024-04-23T01:51:25.619851+0000","last_clean":"2024-04-23T01:51:25.619851+0000","last_became_active":"2024-04-23T01:50:40.443838+0000","last_became_peered":"2024-04-23T01:50:40.443838+0000","last_unstale":"2024-04-23T01:51:25.619851+0000","last_undegraded":"2024-04-23T01:51:25.619851+0000","last_fullsized":"2024-04-23T01:51:25.619851+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":456,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"44'383","reported_seq":408,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:24.837978+0000","last_change":"2024-04-23T01:50:40.451743+0000","last_active":"2024-04-23T01:51:24.837978+0000","last_peered":"2024-04-23T01:51:24.837978+0000","last_clean":"2024-04-23T01:51:24.837978+0000","last_became_active":"2024-04-23T01:50:40.447092+0000","last_became_peered":"2024-04-23T01:50:40.447092+0000","last_unstale":"2024-04-23T01:51:24.837978+0000","last_undegraded":"2024-04-23T01:51:24.837978+0000","last_fullsized":"2024-04-23T01:51:24.837978+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":384,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'338","reported_seq":363,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:26.586959+0000","last_change":"2024-04-23T01:50:40.477557+0000","last_active":"2024-04-23T01:51:26.586959+0000","last_peered":"2024-04-23T01:51:26.586959+0000","last_clean":"2024-04-23T01:51:26.586959+0000","last_became_active":"2024-04-23T01:50:40.472479+0000","last_became_peered":"2024-04-23T01:50:40.472479+0000","last_unstale":"2024-04-23T01:51:26.586959+0000","last_undegraded":"2024-04-23T01:51:26.586959+0000","last_fullsized":"2024-04-23T01:51:26.586959+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2777088,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":339,"num_write_kb":2712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":227,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:13.373808+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:51:13.373808+0000","last_peered":"2024-04-23T01:51:13.373808+0000","last_clean":"2024-04-23T01:51:13.373808+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:51:13.373808+0000","last_undegraded":"2024-04-23T01:51:13.373808+0000","last_fullsized":"2024-04-23T01:51:13.373808+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":94,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:17.623030+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:51:17.623030+0000","last_peered":"2024-04-23T01:51:17.623030+0000","last_clean":"2024-04-23T01:51:17.623030+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:51:17.623030+0000","last_undegraded":"2024-04-23T01:51:17.623030+0000","last_fullsized":"2024-04-23T01:51:17.623030+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"44'375","reported_seq":400,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:26.140599+0000","last_change":"2024-04-23T01:50:40.431365+0000","last_active":"2024-04-23T01:51:26.140599+0000","last_peered":"2024-04-23T01:51:26.140599+0000","last_clean":"2024-04-23T01:51:26.140599+0000","last_became_active":"2024-04-23T01:50:40.426237+0000","last_became_peered":"2024-04-23T01:50:40.426237+0000","last_unstale":"2024-04-23T01:51:26.140599+0000","last_undegraded":"2024-04-23T01:51:26.140599+0000","last_fullsized":"2024-04-23T01:51:26.140599+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":3008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":92,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:16.753715+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:51:16.753715+0000","last_peered":"2024-04-23T01:51:16.753715+0000","last_clean":"2024-04-23T01:51:16.753715+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:51:16.753715+0000","last_undegraded":"2024-04-23T01:51:16.753715+0000","last_fullsized":"2024-04-23T01:51:16.753715+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"44'472","reported_seq":497,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:25.462758+0000","last_change":"2024-04-23T01:50:40.486856+0000","last_active":"2024-04-23T01:51:25.462758+0000","last_peered":"2024-04-23T01:51:25.462758+0000","last_clean":"2024-04-23T01:51:25.462758+0000","last_became_active":"2024-04-23T01:50:40.478453+0000","last_became_peered":"2024-04-23T01:50:40.478453+0000","last_unstale":"2024-04-23T01:51:25.462758+0000","last_undegraded":"2024-04-23T01:51:25.462758+0000","last_fullsized":"2024-04-23T01:51:25.462758+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3874816,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":473,"num_write_kb":3784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":94,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:17.623032+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:51:17.623032+0000","last_peered":"2024-04-23T01:51:17.623032+0000","last_clean":"2024-04-23T01:51:17.623032+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:51:17.623032+0000","last_undegraded":"2024-04-23T01:51:17.623032+0000","last_fullsized":"2024-04-23T01:51:17.623032+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"44'367","reported_seq":392,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:24.393474+0000","last_change":"2024-04-23T01:50:40.462462+0000","last_active":"2024-04-23T01:51:24.393474+0000","last_peered":"2024-04-23T01:51:24.393474+0000","last_clean":"2024-04-23T01:51:24.393474+0000","last_became_active":"2024-04-23T01:50:40.439584+0000","last_became_peered":"2024-04-23T01:50:40.439584+0000","last_unstale":"2024-04-23T01:51:24.393474+0000","last_undegraded":"2024-04-23T01:51:24.393474+0000","last_fullsized":"2024-04-23T01:51:24.393474+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3014656,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":368,"num_write_kb":2944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":92,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:16.754133+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:51:16.754133+0000","last_peered":"2024-04-23T01:51:16.754133+0000","last_clean":"2024-04-23T01:51:16.754133+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:51:16.754133+0000","last_undegraded":"2024-04-23T01:51:16.754133+0000","last_fullsized":"2024-04-23T01:51:16.754133+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"44'471","reported_seq":496,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:26.413199+0000","last_change":"2024-04-23T01:50:40.466468+0000","last_active":"2024-04-23T01:51:26.413199+0000","last_peered":"2024-04-23T01:51:26.413199+0000","last_clean":"2024-04-23T01:51:26.413199+0000","last_became_active":"2024-04-23T01:50:40.452408+0000","last_became_peered":"2024-04-23T01:50:40.452408+0000","last_unstale":"2024-04-23T01:51:26.413199+0000","last_undegraded":"2024-04-23T01:51:26.413199+0000","last_fullsized":"2024-04-23T01:51:26.413199+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3866624,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":472,"num_write_kb":3776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":98,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:16.748257+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:51:16.748257+0000","last_peered":"2024-04-23T01:51:16.748257+0000","last_clean":"2024-04-23T01:51:16.748257+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:51:16.748257+0000","last_undegraded":"2024-04-23T01:51:16.748257+0000","last_fullsized":"2024-04-23T01:51:16.748257+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"44'495","reported_seq":520,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:26.589141+0000","last_change":"2024-04-23T01:50:40.469709+0000","last_active":"2024-04-23T01:51:26.589141+0000","last_peered":"2024-04-23T01:51:26.589141+0000","last_clean":"2024-04-23T01:51:26.589141+0000","last_became_active":"2024-04-23T01:50:40.462062+0000","last_became_peered":"2024-04-23T01:50:40.462062+0000","last_unstale":"2024-04-23T01:51:26.589141+0000","last_undegraded":"2024-04-23T01:51:26.589141+0000","last_fullsized":"2024-04-23T01:51:26.589141+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4063232,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":496,"num_write_kb":3968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":94,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:13.380426+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:51:13.380426+0000","last_peered":"2024-04-23T01:51:13.380426+0000","last_clean":"2024-04-23T01:51:13.380426+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:51:13.380426+0000","last_undegraded":"2024-04-23T01:51:13.380426+0000","last_fullsized":"2024-04-23T01:51:13.380426+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"44'431","reported_seq":456,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:26.302548+0000","last_change":"2024-04-23T01:50:40.464715+0000","last_active":"2024-04-23T01:51:26.302548+0000","last_peered":"2024-04-23T01:51:26.302548+0000","last_clean":"2024-04-23T01:51:26.302548+0000","last_became_active":"2024-04-23T01:50:40.457999+0000","last_became_peered":"2024-04-23T01:50:40.457999+0000","last_unstale":"2024-04-23T01:51:26.302548+0000","last_undegraded":"2024-04-23T01:51:26.302548+0000","last_fullsized":"2024-04-23T01:51:26.302548+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3538944,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":432,"num_write_kb":3456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":98,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:16.747086+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:51:16.747086+0000","last_peered":"2024-04-23T01:51:16.747086+0000","last_clean":"2024-04-23T01:51:16.747086+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:51:16.747086+0000","last_undegraded":"2024-04-23T01:51:16.747086+0000","last_fullsized":"2024-04-23T01:51:16.747086+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"44'479","reported_seq":504,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:26.392874+0000","last_change":"2024-04-23T01:50:40.447151+0000","last_active":"2024-04-23T01:51:26.392874+0000","last_peered":"2024-04-23T01:51:26.392874+0000","last_clean":"2024-04-23T01:51:26.392874+0000","last_became_active":"2024-04-23T01:50:40.441283+0000","last_became_peered":"2024-04-23T01:50:40.441283+0000","last_unstale":"2024-04-23T01:51:26.392874+0000","last_undegraded":"2024-04-23T01:51:26.392874+0000","last_fullsized":"2024-04-23T01:51:26.392874+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":480,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":94,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:17.626169+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:51:17.626169+0000","last_peered":"2024-04-23T01:51:17.626169+0000","last_clean":"2024-04-23T01:51:17.626169+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:51:17.626169+0000","last_undegraded":"2024-04-23T01:51:17.626169+0000","last_fullsized":"2024-04-23T01:51:17.626169+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":54935552,"num_objects":840,"num_object_clones":0,"num_object_copies":2520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6706,"num_write_kb":53648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":82399232,"data_stored":164798464,"data_compressed":8506161,"data_compressed_allocated":82399232,"data_compressed_original":164798464,"omap_allocated":0,"internal_metadata":0},"log_size":6690,"ondisk_log_size":6690,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706043,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":94212,"kb_used_data":18812,"kb_used_omap":0,"kb_used_meta":75392,"kb_avail":94277628,"statfs":{"total":96636764160,"available":96540291072,"internally_reserved":0,"allocated":19263488,"data_stored":38372689,"data_compressed":1973259,"data_compressed_allocated":18866176,"data_compressed_original":38223872,"omap_allocated":0,"internal_metadata":77201408},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706043,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":81924,"kb_used_data":19004,"kb_used_omap":0,"kb_used_meta":62912,"kb_avail":94289916,"statfs":{"total":96636764160,"available":96552873984,"internally_reserved":0,"allocated":19460096,"data_stored":38765905,"data_compressed":1993623,"data_compressed_allocated":19062784,"data_compressed_original":38617088,"omap_allocated":0,"internal_metadata":64421888},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706043,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":88676,"kb_used_data":21596,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94283164,"statfs":{"total":96636764160,"available":96545959936,"internally_reserved":0,"allocated":22114304,"data_stored":43606833,"data_compressed":2243937,"data_compressed_allocated":21737472,"data_compressed_original":43474944,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706043,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":93000,"kb_used_data":21760,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94278840,"statfs":{"total":96636764160,"available":96541532160,"internally_reserved":0,"allocated":22282240,"data_stored":44410193,"data_compressed":2284950,"data_compressed_allocated":21884928,"data_compressed_original":44261376,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22089728,"data_stored":44179456,"data_compressed":2280377,"data_compressed_allocated":22089728,"data_compressed_original":44179456,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22028288,"data_stored":44056576,"data_compressed":2273970,"data_compressed_allocated":22028288,"data_compressed_original":44056576,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19226624,"data_stored":38453248,"data_compressed":1984820,"data_compressed_allocated":19226624,"data_compressed_original":38453248,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19054592,"data_stored":38109184,"data_compressed":1966994,"data_compressed_allocated":19054592,"data_compressed_original":38109184,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:51:32.277 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:51:32.355 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 51 16 7529 7513 1.15079 1.07812 0.0546571 0.108389 2024-04-23T01:51:32.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:32.398+0000 7ff78e26d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:32.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:32.404+0000 7ff78e26d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:33.355 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 52 16 7660 7644 1.14834 1.02344 0.0573693 0.108676 2024-04-23T01:51:34.355 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 53 16 7786 7770 1.14524 0.984375 0.0838517 0.109038 2024-04-23T01:51:34.573 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:51:34.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:34.710+0000 7f6f3006d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:34.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:34.717+0000 7f6f3006d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:35.355 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 54 16 7913 7897 1.14241 0.992188 0.0925286 0.109232 2024-04-23T01:51:36.355 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 55 16 8064 8048 1.14308 1.17969 0.0757488 0.109228 2024-04-23T01:51:36.666 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:51:36.666 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:51:36.666 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:51:36.666 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:51:36.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:36.804+0000 7f890566d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:36.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:36.818+0000 7f890566d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:36.837 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:51:36.838 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:51:36.838 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:51:36.838 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:51:36.838 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:51:36.838 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:51:36.838 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:51:36.839 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:51:36.839 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:51:36.839 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:51:36.839 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:51:36.839 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:51:36.843 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:51:36.843 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:51:36.844 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:51:36.894 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:51:36.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:36.971+0000 7ff0802ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:36.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:36.981+0000 7ff0802ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:37.014 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:51:37.014 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:51:37.014 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:51:37.014 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:51:37.014 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:51:37.014 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:51:37.014 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:51:37.015 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:51:37.015 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:51:37.015 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:51:37.015 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:51:37.015 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:51:37.017 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:51:37.017 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:51:37.017 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:51:37.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:37.039+0000 7f22aeb5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:37.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:37.047+0000 7f22aeb5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:37.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:37.141+0000 7fe1eb35d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:37.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:37.155+0000 7fe1eb35d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:37.181 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:51:37.181 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:51:37.181 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:51:37.181 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:51:37.181 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:51:37.181 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:51:37.181 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:51:37.182 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:51:37.182 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:51:37.182 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:51:37.182 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:51:37.182 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:51:37.184 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:51:37.185 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:51:37.185 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:51:37.355 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 56 16 8196 8180 1.14108 1.03125 0.299284 0.109407 2024-04-23T01:51:37.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:37.351+0000 7fbc8099d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:37.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:37.363+0000 7fbc8099d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:37.411 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:51:37.411 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:51:37.412 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:51:37.412 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:51:37.412 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:51:37.412 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:51:37.412 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:51:37.412 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:51:37.412 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:51:37.412 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:51:37.412 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:51:37.412 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:51:37.414 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:51:37.415 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:51:38.356 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 57 16 8335 8319 1.14012 1.08594 0.0327963 0.109525 2024-04-23T01:51:39.184 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:51:39.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:39.321+0000 7f2db56ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:39.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:39.327+0000 7f2db56ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:39.355 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 58 16 8474 8458 1.13918 1.08594 0.143125 0.109569 2024-04-23T01:51:40.356 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 59 16 8597 8581 1.13616 0.960938 0.131624 0.109872 2024-04-23T01:51:41.356 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:51:41.356554+0000 min lat: 0.0141099 max lat: 0.342036 avg lat: 0.110223 2024-04-23T01:51:41.356 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:51:41.356 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 60 16 8712 8696 1.13219 0.898438 0.282564 0.110223 2024-04-23T01:51:41.525 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:51:41.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:41.643+0000 7eff0d0ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:41.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:41.651+0000 7eff0d0ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:42.356 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 61 16 8843 8827 1.13041 1.02344 0.0835065 0.110364 2024-04-23T01:51:42.415 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:51:42.416 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:51:42.416 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:51:42.416 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:51:42.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:42.544+0000 7f86f72ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:42.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:42.557+0000 7f86f72ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:42.581 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:51:42.581 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:51:42.581 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:51:42.581 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:51:42.581 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:51:42.581 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:51:42.581 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:51:42.581 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:51:42.581 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:51:42.581 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:51:42.582 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:51:42.582 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:51:42.585 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:51:42.585 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:51:42.586 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:51:42.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:42.727+0000 7fd6d659d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:42.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:42.739+0000 7fd6d659d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:42.798 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:51:42.798 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:51:42.798 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:51:42.798 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:51:42.798 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:51:42.798 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:51:42.798 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:51:42.798 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:51:42.798 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:51:42.799 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:51:42.799 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:51:42.799 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:51:42.802 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:51:42.802 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:51:42.802 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:51:42.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:42.931+0000 7fc9576ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:42.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:42.940+0000 7fc9576ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:42.958 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:51:42.958 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:51:42.958 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:51:42.958 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:51:42.958 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:51:42.958 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:51:42.958 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:51:42.958 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:51:42.958 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:51:42.959 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:51:42.959 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:51:42.959 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:51:42.961 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:51:42.962 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:51:42.962 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:51:43.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:43.084+0000 7f2ebc66d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:43.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:43.096+0000 7f2ebc66d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:43.130 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:51:43.130 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:51:43.130 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:51:43.130 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:51:43.130 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:51:43.130 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:51:43.130 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:51:43.131 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:51:43.131 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:51:43.131 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:51:43.131 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:51:43.131 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:51:43.133 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:51:43.133 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:51:43.356 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 62 16 8984 8968 1.12994 1.10156 0.106191 0.110473 2024-04-23T01:51:43.804 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:51:43.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:43.928+0000 7f4d3179d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:43.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:43.935+0000 7f4d3179d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:44.356 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 63 16 9121 9105 1.12899 1.07031 0.0344692 0.11061 2024-04-23T01:51:45.358 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 64 16 9248 9232 1.12685 0.992188 0.100269 0.110757 2024-04-23T01:51:46.111 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:51:46.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:46.235+0000 7fd260464640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:46.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:46.243+0000 7fd260464640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:46.356 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 65 16 9384 9368 1.12586 1.0625 0.183161 0.110888 2024-04-23T01:51:47.356 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 66 16 9530 9514 1.12609 1.14062 0.08415 0.11088 2024-04-23T01:51:48.134 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:51:48.135 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:51:48.135 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:51:48.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:48.273+0000 7fe4e2c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:48.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:48.282+0000 7fe4e2c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:48.356 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 67 16 9674 9658 1.12607 1.125 0.126519 0.110901 2024-04-23T01:51:48.431 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:51:48.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:48.558+0000 7f186575d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:48.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:48.568+0000 7f186575d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:48.659 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:51:48.660 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:51:48.677 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":238,"stamp":"2024-04-23T01:51:47.625415+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":77685280,"num_objects":1180,"num_object_clones":0,"num_object_copies":3540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":9548,"num_write_kb":77182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9492,"ondisk_log_size":9492,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":390364,"kb_used_data":113720,"kb_used_omap":0,"kb_used_meta":276608,"kb_avail":377096996,"statfs":{"total":386547056640,"available":386147323904,"internally_reserved":0,"allocated":116449280,"data_stored":231813924,"data_compressed":11937720,"data_compressed_allocated":114880512,"data_compressed_original":231235584,"omap_allocated":0,"internal_metadata":283246592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16334848,"num_objects":250,"num_object_clones":0,"num_object_copies":750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1994,"num_write_kb":15952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002143"},"pg_stats":[{"pgid":"6.f","version":"44'583","reported_seq":608,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:43.675452+0000","last_change":"2024-04-23T01:50:40.440701+0000","last_active":"2024-04-23T01:51:43.675452+0000","last_peered":"2024-04-23T01:51:43.675452+0000","last_clean":"2024-04-23T01:51:43.675452+0000","last_became_active":"2024-04-23T01:50:40.433832+0000","last_became_peered":"2024-04-23T01:50:40.433832+0000","last_unstale":"2024-04-23T01:51:43.675452+0000","last_undegraded":"2024-04-23T01:51:43.675452+0000","last_fullsized":"2024-04-23T01:51:43.675452+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4784128,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":584,"num_write_kb":4672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"44'519","reported_seq":544,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:46.606803+0000","last_change":"2024-04-23T01:50:40.477278+0000","last_active":"2024-04-23T01:51:46.606803+0000","last_peered":"2024-04-23T01:51:46.606803+0000","last_clean":"2024-04-23T01:51:46.606803+0000","last_became_active":"2024-04-23T01:50:40.471274+0000","last_became_peered":"2024-04-23T01:50:40.471274+0000","last_unstale":"2024-04-23T01:51:46.606803+0000","last_undegraded":"2024-04-23T01:51:46.606803+0000","last_fullsized":"2024-04-23T01:51:46.606803+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259840,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":520,"num_write_kb":4160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"44'583","reported_seq":608,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:44.618176+0000","last_change":"2024-04-23T01:50:40.446224+0000","last_active":"2024-04-23T01:51:44.618176+0000","last_peered":"2024-04-23T01:51:44.618176+0000","last_clean":"2024-04-23T01:51:44.618176+0000","last_became_active":"2024-04-23T01:50:40.437838+0000","last_became_peered":"2024-04-23T01:50:40.437838+0000","last_unstale":"2024-04-23T01:51:44.618176+0000","last_undegraded":"2024-04-23T01:51:44.618176+0000","last_fullsized":"2024-04-23T01:51:44.618176+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4784128,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":584,"num_write_kb":4672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"44'535","reported_seq":560,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:45.609590+0000","last_change":"2024-04-23T01:50:40.517607+0000","last_active":"2024-04-23T01:51:45.609590+0000","last_peered":"2024-04-23T01:51:45.609590+0000","last_clean":"2024-04-23T01:51:45.609590+0000","last_became_active":"2024-04-23T01:50:40.511881+0000","last_became_peered":"2024-04-23T01:50:40.511881+0000","last_unstale":"2024-04-23T01:51:45.609590+0000","last_undegraded":"2024-04-23T01:51:45.609590+0000","last_fullsized":"2024-04-23T01:51:45.609590+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4390912,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":536,"num_write_kb":4288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'767","reported_seq":792,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:46.575475+0000","last_change":"2024-04-23T01:50:40.432817+0000","last_active":"2024-04-23T01:51:46.575475+0000","last_peered":"2024-04-23T01:51:46.575475+0000","last_clean":"2024-04-23T01:51:46.575475+0000","last_became_active":"2024-04-23T01:50:40.421722+0000","last_became_peered":"2024-04-23T01:50:40.421722+0000","last_unstale":"2024-04-23T01:51:46.575475+0000","last_undegraded":"2024-04-23T01:51:46.575475+0000","last_fullsized":"2024-04-23T01:51:46.575475+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":768,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"44'439","reported_seq":464,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:45.699844+0000","last_change":"2024-04-23T01:50:40.444510+0000","last_active":"2024-04-23T01:51:45.699844+0000","last_peered":"2024-04-23T01:51:45.699844+0000","last_clean":"2024-04-23T01:51:45.699844+0000","last_became_active":"2024-04-23T01:50:40.437088+0000","last_became_peered":"2024-04-23T01:50:40.437088+0000","last_unstale":"2024-04-23T01:51:45.699844+0000","last_undegraded":"2024-04-23T01:51:45.699844+0000","last_fullsized":"2024-04-23T01:51:45.699844+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3604480,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":440,"num_write_kb":3520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"44'559","reported_seq":584,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:46.500103+0000","last_change":"2024-04-23T01:50:40.468123+0000","last_active":"2024-04-23T01:51:46.500103+0000","last_peered":"2024-04-23T01:51:46.500103+0000","last_clean":"2024-04-23T01:51:46.500103+0000","last_became_active":"2024-04-23T01:50:40.443838+0000","last_became_peered":"2024-04-23T01:50:40.443838+0000","last_unstale":"2024-04-23T01:51:46.500103+0000","last_undegraded":"2024-04-23T01:51:46.500103+0000","last_fullsized":"2024-04-23T01:51:46.500103+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4587520,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":560,"num_write_kb":4480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"44'503","reported_seq":528,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:44.318471+0000","last_change":"2024-04-23T01:50:40.451743+0000","last_active":"2024-04-23T01:51:44.318471+0000","last_peered":"2024-04-23T01:51:44.318471+0000","last_clean":"2024-04-23T01:51:44.318471+0000","last_became_active":"2024-04-23T01:50:40.447092+0000","last_became_peered":"2024-04-23T01:50:40.447092+0000","last_unstale":"2024-04-23T01:51:44.318471+0000","last_undegraded":"2024-04-23T01:51:44.318471+0000","last_fullsized":"2024-04-23T01:51:44.318471+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4128768,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":504,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'527","reported_seq":552,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:44.005133+0000","last_change":"2024-04-23T01:50:40.477557+0000","last_active":"2024-04-23T01:51:44.005133+0000","last_peered":"2024-04-23T01:51:44.005133+0000","last_clean":"2024-04-23T01:51:44.005133+0000","last_became_active":"2024-04-23T01:50:40.472479+0000","last_became_peered":"2024-04-23T01:50:40.472479+0000","last_unstale":"2024-04-23T01:51:44.005133+0000","last_undegraded":"2024-04-23T01:51:44.005133+0000","last_fullsized":"2024-04-23T01:51:44.005133+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":528,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":227,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:13.373808+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:51:13.373808+0000","last_peered":"2024-04-23T01:51:13.373808+0000","last_clean":"2024-04-23T01:51:13.373808+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:51:13.373808+0000","last_undegraded":"2024-04-23T01:51:13.373808+0000","last_fullsized":"2024-04-23T01:51:13.373808+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":94,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:17.623030+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:51:17.623030+0000","last_peered":"2024-04-23T01:51:17.623030+0000","last_clean":"2024-04-23T01:51:17.623030+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:51:17.623030+0000","last_undegraded":"2024-04-23T01:51:17.623030+0000","last_fullsized":"2024-04-23T01:51:17.623030+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"44'599","reported_seq":624,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:45.646294+0000","last_change":"2024-04-23T01:50:40.431365+0000","last_active":"2024-04-23T01:51:45.646294+0000","last_peered":"2024-04-23T01:51:45.646294+0000","last_clean":"2024-04-23T01:51:45.646294+0000","last_became_active":"2024-04-23T01:50:40.426237+0000","last_became_peered":"2024-04-23T01:50:40.426237+0000","last_unstale":"2024-04-23T01:51:45.646294+0000","last_undegraded":"2024-04-23T01:51:45.646294+0000","last_fullsized":"2024-04-23T01:51:45.646294+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":92,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:16.753715+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:51:16.753715+0000","last_peered":"2024-04-23T01:51:16.753715+0000","last_clean":"2024-04-23T01:51:16.753715+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:51:16.753715+0000","last_undegraded":"2024-04-23T01:51:16.753715+0000","last_fullsized":"2024-04-23T01:51:16.753715+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"44'661","reported_seq":686,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:46.578554+0000","last_change":"2024-04-23T01:50:40.486856+0000","last_active":"2024-04-23T01:51:46.578554+0000","last_peered":"2024-04-23T01:51:46.578554+0000","last_clean":"2024-04-23T01:51:46.578554+0000","last_became_active":"2024-04-23T01:50:40.478453+0000","last_became_peered":"2024-04-23T01:50:40.478453+0000","last_unstale":"2024-04-23T01:51:46.578554+0000","last_undegraded":"2024-04-23T01:51:46.578554+0000","last_fullsized":"2024-04-23T01:51:46.578554+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5423104,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":662,"num_write_kb":5296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":94,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:17.623032+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:51:17.623032+0000","last_peered":"2024-04-23T01:51:17.623032+0000","last_clean":"2024-04-23T01:51:17.623032+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:51:17.623032+0000","last_undegraded":"2024-04-23T01:51:17.623032+0000","last_fullsized":"2024-04-23T01:51:17.623032+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"44'559","reported_seq":584,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:46.129627+0000","last_change":"2024-04-23T01:50:40.462462+0000","last_active":"2024-04-23T01:51:46.129627+0000","last_peered":"2024-04-23T01:51:46.129627+0000","last_clean":"2024-04-23T01:51:46.129627+0000","last_became_active":"2024-04-23T01:50:40.439584+0000","last_became_peered":"2024-04-23T01:50:40.439584+0000","last_unstale":"2024-04-23T01:51:46.129627+0000","last_undegraded":"2024-04-23T01:51:46.129627+0000","last_fullsized":"2024-04-23T01:51:46.129627+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4587520,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":560,"num_write_kb":4480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":92,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:16.754133+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:51:16.754133+0000","last_peered":"2024-04-23T01:51:16.754133+0000","last_clean":"2024-04-23T01:51:16.754133+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:51:16.754133+0000","last_undegraded":"2024-04-23T01:51:16.754133+0000","last_fullsized":"2024-04-23T01:51:16.754133+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"44'632","reported_seq":657,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:45.181173+0000","last_change":"2024-04-23T01:50:40.466468+0000","last_active":"2024-04-23T01:51:45.181173+0000","last_peered":"2024-04-23T01:51:45.181173+0000","last_clean":"2024-04-23T01:51:45.181173+0000","last_became_active":"2024-04-23T01:50:40.452408+0000","last_became_peered":"2024-04-23T01:50:40.452408+0000","last_unstale":"2024-04-23T01:51:45.181173+0000","last_undegraded":"2024-04-23T01:51:45.181173+0000","last_fullsized":"2024-04-23T01:51:45.181173+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5185536,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":633,"num_write_kb":5064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":98,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:16.748257+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:51:16.748257+0000","last_peered":"2024-04-23T01:51:16.748257+0000","last_clean":"2024-04-23T01:51:16.748257+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:51:16.748257+0000","last_undegraded":"2024-04-23T01:51:16.748257+0000","last_fullsized":"2024-04-23T01:51:16.748257+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"44'695","reported_seq":720,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:45.896268+0000","last_change":"2024-04-23T01:50:40.469709+0000","last_active":"2024-04-23T01:51:45.896268+0000","last_peered":"2024-04-23T01:51:45.896268+0000","last_clean":"2024-04-23T01:51:45.896268+0000","last_became_active":"2024-04-23T01:50:40.462062+0000","last_became_peered":"2024-04-23T01:50:40.462062+0000","last_unstale":"2024-04-23T01:51:45.896268+0000","last_undegraded":"2024-04-23T01:51:45.896268+0000","last_fullsized":"2024-04-23T01:51:45.896268+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":696,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":94,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:13.380426+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:51:13.380426+0000","last_peered":"2024-04-23T01:51:13.380426+0000","last_clean":"2024-04-23T01:51:13.380426+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:51:13.380426+0000","last_undegraded":"2024-04-23T01:51:13.380426+0000","last_fullsized":"2024-04-23T01:51:13.380426+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"44'615","reported_seq":640,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:46.386863+0000","last_change":"2024-04-23T01:50:40.464715+0000","last_active":"2024-04-23T01:51:46.386863+0000","last_peered":"2024-04-23T01:51:46.386863+0000","last_clean":"2024-04-23T01:51:46.386863+0000","last_became_active":"2024-04-23T01:50:40.457999+0000","last_became_peered":"2024-04-23T01:50:40.457999+0000","last_unstale":"2024-04-23T01:51:46.386863+0000","last_undegraded":"2024-04-23T01:51:46.386863+0000","last_fullsized":"2024-04-23T01:51:46.386863+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5046272,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":616,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":98,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:16.747086+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:51:16.747086+0000","last_peered":"2024-04-23T01:51:16.747086+0000","last_clean":"2024-04-23T01:51:16.747086+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:51:16.747086+0000","last_undegraded":"2024-04-23T01:51:16.747086+0000","last_fullsized":"2024-04-23T01:51:16.747086+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"44'619","reported_seq":644,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:46.603919+0000","last_change":"2024-04-23T01:50:40.447151+0000","last_active":"2024-04-23T01:51:46.603919+0000","last_peered":"2024-04-23T01:51:46.603919+0000","last_clean":"2024-04-23T01:51:46.603919+0000","last_became_active":"2024-04-23T01:50:40.441283+0000","last_became_peered":"2024-04-23T01:50:40.441283+0000","last_unstale":"2024-04-23T01:51:46.603919+0000","last_undegraded":"2024-04-23T01:51:46.603919+0000","last_fullsized":"2024-04-23T01:51:46.603919+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5079040,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":620,"num_write_kb":4960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":94,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:17.626169+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:51:17.626169+0000","last_peered":"2024-04-23T01:51:17.626169+0000","last_clean":"2024-04-23T01:51:17.626169+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:51:17.626169+0000","last_undegraded":"2024-04-23T01:51:17.626169+0000","last_fullsized":"2024-04-23T01:51:17.626169+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":77094912,"num_objects":1178,"num_object_clones":0,"num_object_copies":3534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9411,"num_write_kb":75288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":115638272,"data_stored":231276544,"data_compressed":11938806,"data_compressed_allocated":115638272,"data_compressed_original":231276544,"omap_allocated":0,"internal_metadata":0},"log_size":9395,"ondisk_log_size":9395,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706063,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":93556,"kb_used_data":26476,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94278284,"statfs":{"total":96636764160,"available":96540962816,"internally_reserved":0,"allocated":27111424,"data_stored":54068561,"data_compressed":2783727,"data_compressed_allocated":26714112,"data_compressed_original":53919744,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706063,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":97468,"kb_used_data":26228,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94274372,"statfs":{"total":96636764160,"available":96536956928,"internally_reserved":0,"allocated":26857472,"data_stored":53560657,"data_compressed":2757561,"data_compressed_allocated":26460160,"data_compressed_original":53411840,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706063,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":105964,"kb_used_data":30564,"kb_used_omap":0,"kb_used_meta":75392,"kb_avail":94265876,"statfs":{"total":96636764160,"available":96528257024,"internally_reserved":0,"allocated":31297536,"data_stored":61973297,"data_compressed":3192303,"data_compressed_allocated":30920704,"data_compressed_original":61841408,"omap_allocated":0,"internal_metadata":77201408},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706063,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":93376,"kb_used_data":30452,"kb_used_omap":0,"kb_used_meta":62912,"kb_avail":94278464,"statfs":{"total":96636764160,"available":96541147136,"internally_reserved":0,"allocated":31182848,"data_stored":62211409,"data_compressed":3204129,"data_compressed_allocated":30785536,"data_compressed_original":62062592,"omap_allocated":0,"internal_metadata":64421888},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":30908416,"data_stored":61816832,"data_compressed":3191096,"data_compressed_allocated":30908416,"data_compressed_original":61816832,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":31178752,"data_stored":62357504,"data_compressed":3218952,"data_compressed_allocated":31178752,"data_compressed_original":62357504,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":26701824,"data_stored":53403648,"data_compressed":2756795,"data_compressed_allocated":26701824,"data_compressed_original":53403648,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":26849280,"data_stored":53698560,"data_compressed":2771963,"data_compressed_allocated":26849280,"data_compressed_original":53698560,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:51:49.356 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 68 16 9797 9781 1.12364 0.960938 0.150115 0.111137 2024-04-23T01:51:50.357 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 69 16 9928 9912 1.12219 1.02344 0.132153 0.111258 2024-04-23T01:51:50.783 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:51:50.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:50.914+0000 7f68dc46d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:50.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:50.920+0000 7f68dc46d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:51.357 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 70 16 10061 10045 1.121 1.03906 0.160672 0.111398 2024-04-23T01:51:52.357 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 71 16 10197 10181 1.12017 1.0625 0.0984062 0.111464 2024-04-23T01:51:53.087 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:51:53.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:53.238+0000 7f37e36ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:53.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:53.246+0000 7f37e36ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:53.357 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 72 16 10330 10314 1.11904 1.03906 0.167917 0.111557 2024-04-23T01:51:53.680 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:51:53.680 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:51:53.680 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:51:53.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:53.831+0000 7fccf9064640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:53.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:53.840+0000 7fccf9064640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:54.209 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:51:54.210 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:51:54.232 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":241,"stamp":"2024-04-23T01:51:53.626532+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":83247648,"num_objects":1265,"num_object_clones":0,"num_object_copies":3795,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":10227,"num_write_kb":82614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10171,"ondisk_log_size":10171,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":406808,"kb_used_data":121848,"kb_used_omap":0,"kb_used_meta":284928,"kb_avail":377080552,"statfs":{"total":386547056640,"available":386130485248,"internally_reserved":0,"allocated":124772352,"data_stored":248460068,"data_compressed":12797256,"data_compressed_allocated":123203584,"data_compressed_original":247881728,"omap_allocated":0,"internal_metadata":291766272},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16384000,"num_objects":251,"num_object_clones":0,"num_object_copies":753,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2000,"num_write_kb":16000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002207"},"pg_stats":[{"pgid":"6.f","version":"44'612","reported_seq":637,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:51.596766+0000","last_change":"2024-04-23T01:50:40.440701+0000","last_active":"2024-04-23T01:51:51.596766+0000","last_peered":"2024-04-23T01:51:51.596766+0000","last_clean":"2024-04-23T01:51:51.596766+0000","last_became_active":"2024-04-23T01:50:40.433832+0000","last_became_peered":"2024-04-23T01:50:40.433832+0000","last_unstale":"2024-04-23T01:51:51.596766+0000","last_undegraded":"2024-04-23T01:51:51.596766+0000","last_fullsized":"2024-04-23T01:51:51.596766+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":612,"log_dups_size":0,"ondisk_log_size":612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5021696,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":613,"num_write_kb":4904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"44'567","reported_seq":592,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:51.460681+0000","last_change":"2024-04-23T01:50:40.477278+0000","last_active":"2024-04-23T01:51:51.460681+0000","last_peered":"2024-04-23T01:51:51.460681+0000","last_clean":"2024-04-23T01:51:51.460681+0000","last_became_active":"2024-04-23T01:50:40.471274+0000","last_became_peered":"2024-04-23T01:50:40.471274+0000","last_unstale":"2024-04-23T01:51:51.460681+0000","last_undegraded":"2024-04-23T01:51:51.460681+0000","last_fullsized":"2024-04-23T01:51:51.460681+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":568,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"44'615","reported_seq":640,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:51.193147+0000","last_change":"2024-04-23T01:50:40.446224+0000","last_active":"2024-04-23T01:51:51.193147+0000","last_peered":"2024-04-23T01:51:51.193147+0000","last_clean":"2024-04-23T01:51:51.193147+0000","last_became_active":"2024-04-23T01:50:40.437838+0000","last_became_peered":"2024-04-23T01:50:40.437838+0000","last_unstale":"2024-04-23T01:51:51.193147+0000","last_undegraded":"2024-04-23T01:51:51.193147+0000","last_fullsized":"2024-04-23T01:51:51.193147+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5046272,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":616,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"44'583","reported_seq":608,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:50.661246+0000","last_change":"2024-04-23T01:50:40.517607+0000","last_active":"2024-04-23T01:51:50.661246+0000","last_peered":"2024-04-23T01:51:50.661246+0000","last_clean":"2024-04-23T01:51:50.661246+0000","last_became_active":"2024-04-23T01:50:40.511881+0000","last_became_peered":"2024-04-23T01:50:40.511881+0000","last_unstale":"2024-04-23T01:51:50.661246+0000","last_undegraded":"2024-04-23T01:51:50.661246+0000","last_fullsized":"2024-04-23T01:51:50.661246+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4784128,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":584,"num_write_kb":4672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'815","reported_seq":840,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:51.083197+0000","last_change":"2024-04-23T01:50:40.432817+0000","last_active":"2024-04-23T01:51:51.083197+0000","last_peered":"2024-04-23T01:51:51.083197+0000","last_clean":"2024-04-23T01:51:51.083197+0000","last_became_active":"2024-04-23T01:50:40.421722+0000","last_became_peered":"2024-04-23T01:50:40.421722+0000","last_unstale":"2024-04-23T01:51:51.083197+0000","last_undegraded":"2024-04-23T01:51:51.083197+0000","last_fullsized":"2024-04-23T01:51:51.083197+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6684672,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":816,"num_write_kb":6528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"44'471","reported_seq":496,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:51.326672+0000","last_change":"2024-04-23T01:50:40.444510+0000","last_active":"2024-04-23T01:51:51.326672+0000","last_peered":"2024-04-23T01:51:51.326672+0000","last_clean":"2024-04-23T01:51:51.326672+0000","last_became_active":"2024-04-23T01:50:40.437088+0000","last_became_peered":"2024-04-23T01:50:40.437088+0000","last_unstale":"2024-04-23T01:51:51.326672+0000","last_undegraded":"2024-04-23T01:51:51.326672+0000","last_fullsized":"2024-04-23T01:51:51.326672+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3866624,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":472,"num_write_kb":3776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"44'613","reported_seq":638,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:51.590483+0000","last_change":"2024-04-23T01:50:40.468123+0000","last_active":"2024-04-23T01:51:51.590483+0000","last_peered":"2024-04-23T01:51:51.590483+0000","last_clean":"2024-04-23T01:51:51.590483+0000","last_became_active":"2024-04-23T01:50:40.443838+0000","last_became_peered":"2024-04-23T01:50:40.443838+0000","last_unstale":"2024-04-23T01:51:51.590483+0000","last_undegraded":"2024-04-23T01:51:51.590483+0000","last_fullsized":"2024-04-23T01:51:51.590483+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5029888,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":614,"num_write_kb":4912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"44'549","reported_seq":574,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:51.604627+0000","last_change":"2024-04-23T01:50:40.451743+0000","last_active":"2024-04-23T01:51:51.604627+0000","last_peered":"2024-04-23T01:51:51.604627+0000","last_clean":"2024-04-23T01:51:51.604627+0000","last_became_active":"2024-04-23T01:50:40.447092+0000","last_became_peered":"2024-04-23T01:50:40.447092+0000","last_unstale":"2024-04-23T01:51:51.604627+0000","last_undegraded":"2024-04-23T01:51:51.604627+0000","last_fullsized":"2024-04-23T01:51:51.604627+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4505600,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":550,"num_write_kb":4400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'543","reported_seq":568,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:47.223775+0000","last_change":"2024-04-23T01:50:40.477557+0000","last_active":"2024-04-23T01:51:47.223775+0000","last_peered":"2024-04-23T01:51:47.223775+0000","last_clean":"2024-04-23T01:51:47.223775+0000","last_became_active":"2024-04-23T01:50:40.472479+0000","last_became_peered":"2024-04-23T01:50:40.472479+0000","last_unstale":"2024-04-23T01:51:47.223775+0000","last_undegraded":"2024-04-23T01:51:47.223775+0000","last_fullsized":"2024-04-23T01:51:47.223775+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":544,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":227,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:13.373808+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:51:13.373808+0000","last_peered":"2024-04-23T01:51:13.373808+0000","last_clean":"2024-04-23T01:51:13.373808+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:51:13.373808+0000","last_undegraded":"2024-04-23T01:51:13.373808+0000","last_fullsized":"2024-04-23T01:51:13.373808+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":94,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:17.623030+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:51:17.623030+0000","last_peered":"2024-04-23T01:51:17.623030+0000","last_clean":"2024-04-23T01:51:17.623030+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:51:17.623030+0000","last_undegraded":"2024-04-23T01:51:17.623030+0000","last_fullsized":"2024-04-23T01:51:17.623030+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"44'623","reported_seq":648,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:50.158559+0000","last_change":"2024-04-23T01:50:40.431365+0000","last_active":"2024-04-23T01:51:50.158559+0000","last_peered":"2024-04-23T01:51:50.158559+0000","last_clean":"2024-04-23T01:51:50.158559+0000","last_became_active":"2024-04-23T01:50:40.426237+0000","last_became_peered":"2024-04-23T01:50:40.426237+0000","last_unstale":"2024-04-23T01:51:50.158559+0000","last_undegraded":"2024-04-23T01:51:50.158559+0000","last_fullsized":"2024-04-23T01:51:50.158559+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5111808,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":624,"num_write_kb":4992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":92,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:16.753715+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:51:16.753715+0000","last_peered":"2024-04-23T01:51:16.753715+0000","last_clean":"2024-04-23T01:51:16.753715+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:51:16.753715+0000","last_undegraded":"2024-04-23T01:51:16.753715+0000","last_fullsized":"2024-04-23T01:51:16.753715+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"44'760","reported_seq":785,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:50.868014+0000","last_change":"2024-04-23T01:50:40.486856+0000","last_active":"2024-04-23T01:51:50.868014+0000","last_peered":"2024-04-23T01:51:50.868014+0000","last_clean":"2024-04-23T01:51:50.868014+0000","last_became_active":"2024-04-23T01:50:40.478453+0000","last_became_peered":"2024-04-23T01:50:40.478453+0000","last_unstale":"2024-04-23T01:51:50.868014+0000","last_undegraded":"2024-04-23T01:51:50.868014+0000","last_fullsized":"2024-04-23T01:51:50.868014+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":760,"log_dups_size":0,"ondisk_log_size":760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6234112,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":761,"num_write_kb":6088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":94,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:17.623032+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:51:17.623032+0000","last_peered":"2024-04-23T01:51:17.623032+0000","last_clean":"2024-04-23T01:51:17.623032+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:51:17.623032+0000","last_undegraded":"2024-04-23T01:51:17.623032+0000","last_fullsized":"2024-04-23T01:51:17.623032+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"44'599","reported_seq":624,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:51.534955+0000","last_change":"2024-04-23T01:50:40.462462+0000","last_active":"2024-04-23T01:51:51.534955+0000","last_peered":"2024-04-23T01:51:51.534955+0000","last_clean":"2024-04-23T01:51:51.534955+0000","last_became_active":"2024-04-23T01:50:40.439584+0000","last_became_peered":"2024-04-23T01:50:40.439584+0000","last_unstale":"2024-04-23T01:51:51.534955+0000","last_undegraded":"2024-04-23T01:51:51.534955+0000","last_fullsized":"2024-04-23T01:51:51.534955+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":92,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:16.754133+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:51:16.754133+0000","last_peered":"2024-04-23T01:51:16.754133+0000","last_clean":"2024-04-23T01:51:16.754133+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:51:16.754133+0000","last_undegraded":"2024-04-23T01:51:16.754133+0000","last_fullsized":"2024-04-23T01:51:16.754133+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"44'687","reported_seq":712,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:51.477531+0000","last_change":"2024-04-23T01:50:40.466468+0000","last_active":"2024-04-23T01:51:51.477531+0000","last_peered":"2024-04-23T01:51:51.477531+0000","last_clean":"2024-04-23T01:51:51.477531+0000","last_became_active":"2024-04-23T01:50:40.452408+0000","last_became_peered":"2024-04-23T01:50:40.452408+0000","last_unstale":"2024-04-23T01:51:51.477531+0000","last_undegraded":"2024-04-23T01:51:51.477531+0000","last_fullsized":"2024-04-23T01:51:51.477531+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5636096,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":688,"num_write_kb":5504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":98,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:16.748257+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:51:16.748257+0000","last_peered":"2024-04-23T01:51:16.748257+0000","last_clean":"2024-04-23T01:51:16.748257+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:51:16.748257+0000","last_undegraded":"2024-04-23T01:51:16.748257+0000","last_fullsized":"2024-04-23T01:51:16.748257+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"44'727","reported_seq":752,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:51.191743+0000","last_change":"2024-04-23T01:50:40.469709+0000","last_active":"2024-04-23T01:51:51.191743+0000","last_peered":"2024-04-23T01:51:51.191743+0000","last_clean":"2024-04-23T01:51:51.191743+0000","last_became_active":"2024-04-23T01:50:40.462062+0000","last_became_peered":"2024-04-23T01:50:40.462062+0000","last_unstale":"2024-04-23T01:51:51.191743+0000","last_undegraded":"2024-04-23T01:51:51.191743+0000","last_fullsized":"2024-04-23T01:51:51.191743+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":728,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":94,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:13.380426+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:51:13.380426+0000","last_peered":"2024-04-23T01:51:13.380426+0000","last_clean":"2024-04-23T01:51:13.380426+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:51:13.380426+0000","last_undegraded":"2024-04-23T01:51:13.380426+0000","last_fullsized":"2024-04-23T01:51:13.380426+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"44'655","reported_seq":680,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:51.375485+0000","last_change":"2024-04-23T01:50:40.464715+0000","last_active":"2024-04-23T01:51:51.375485+0000","last_peered":"2024-04-23T01:51:51.375485+0000","last_clean":"2024-04-23T01:51:51.375485+0000","last_became_active":"2024-04-23T01:50:40.457999+0000","last_became_peered":"2024-04-23T01:50:40.457999+0000","last_unstale":"2024-04-23T01:51:51.375485+0000","last_undegraded":"2024-04-23T01:51:51.375485+0000","last_fullsized":"2024-04-23T01:51:51.375485+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":98,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:16.747086+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:51:16.747086+0000","last_peered":"2024-04-23T01:51:16.747086+0000","last_clean":"2024-04-23T01:51:16.747086+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:51:16.747086+0000","last_undegraded":"2024-04-23T01:51:16.747086+0000","last_fullsized":"2024-04-23T01:51:16.747086+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"44'655","reported_seq":680,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:50.527127+0000","last_change":"2024-04-23T01:50:40.447151+0000","last_active":"2024-04-23T01:51:50.527127+0000","last_peered":"2024-04-23T01:51:50.527127+0000","last_clean":"2024-04-23T01:51:50.527127+0000","last_became_active":"2024-04-23T01:50:40.441283+0000","last_became_peered":"2024-04-23T01:50:40.441283+0000","last_unstale":"2024-04-23T01:51:50.527127+0000","last_undegraded":"2024-04-23T01:51:50.527127+0000","last_fullsized":"2024-04-23T01:51:50.527127+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":94,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:17.626169+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:51:17.626169+0000","last_peered":"2024-04-23T01:51:17.626169+0000","last_clean":"2024-04-23T01:51:17.626169+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:51:17.626169+0000","last_undegraded":"2024-04-23T01:51:17.626169+0000","last_fullsized":"2024-04-23T01:51:17.626169+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":82657280,"num_objects":1263,"num_object_clones":0,"num_object_copies":3789,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10090,"num_write_kb":80720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":123994112,"data_stored":247988224,"data_compressed":12801726,"data_compressed_allocated":123994112,"data_compressed_original":247988224,"omap_allocated":0,"internal_metadata":0},"log_size":10074,"ondisk_log_size":10074,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706068,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":95536,"kb_used_data":28456,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94276304,"statfs":{"total":96636764160,"available":96538935296,"internally_reserved":0,"allocated":29138944,"data_stored":58123601,"data_compressed":2993112,"data_compressed_allocated":28741632,"data_compressed_original":57974784,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706068,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":103804,"kb_used_data":28404,"kb_used_omap":0,"kb_used_meta":75392,"kb_avail":94268036,"statfs":{"total":96636764160,"available":96530468864,"internally_reserved":0,"allocated":29085696,"data_stored":58017105,"data_compressed":2987673,"data_compressed_allocated":28688384,"data_compressed_original":57868288,"omap_allocated":0,"internal_metadata":77201408},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706068,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":112056,"kb_used_data":32496,"kb_used_omap":0,"kb_used_meta":79552,"kb_avail":94259784,"statfs":{"total":96636764160,"available":96522018816,"internally_reserved":0,"allocated":33275904,"data_stored":65930033,"data_compressed":3396612,"data_compressed_allocated":32899072,"data_compressed_original":65798144,"omap_allocated":0,"internal_metadata":81461248},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706068,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":95412,"kb_used_data":32492,"kb_used_omap":0,"kb_used_meta":62912,"kb_avail":94276428,"statfs":{"total":96636764160,"available":96539062272,"internally_reserved":0,"allocated":33271808,"data_stored":66389329,"data_compressed":3419859,"data_compressed_allocated":32874496,"data_compressed_original":66240512,"omap_allocated":0,"internal_metadata":64421888},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":33079296,"data_stored":66158592,"data_compressed":3415286,"data_compressed_allocated":33079296,"data_compressed_original":66158592,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33144832,"data_stored":66289664,"data_compressed":3421992,"data_compressed_allocated":33144832,"data_compressed_original":66289664,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28880896,"data_stored":57761792,"data_compressed":2981831,"data_compressed_allocated":28880896,"data_compressed_original":57761792,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28889088,"data_stored":57778176,"data_compressed":2982617,"data_compressed_allocated":28889088,"data_compressed_original":57778176,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:51:54.357 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 73 16 10445 10429 1.11602 0.898438 0.0894144 0.111891 2024-04-23T01:51:55.357 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 74 16 10582 10566 1.1154 1.07031 0.0969481 0.111976 2024-04-23T01:51:55.391 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:51:55.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:55.517+0000 7f3ac3aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:55.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:55.525+0000 7f3ac3aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:56.357 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 75 16 10726 10710 1.11553 1.125 0.212566 0.111947 2024-04-23T01:51:57.357 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 76 16 10867 10851 1.11534 1.10156 0.127522 0.111994 2024-04-23T01:51:57.655 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:51:57.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:57.781+0000 7fda2fa6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:57.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:57.789+0000 7fda2fa6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:58.357 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 77 16 11002 10986 1.11455 1.05469 0.0900464 0.112031 2024-04-23T01:51:59.233 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:51:59.234 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:51:59.234 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:51:59.357 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 78 16 11131 11115 1.11318 1.00781 0.131363 0.112188 2024-04-23T01:51:59.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:59.361+0000 7f170df5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:59.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:51:59.372+0000 7f170df5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:51:59.650 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:51:59.651 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:51:59.669 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2024-04-23T01:51:59.627901+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":88695328,"num_objects":1347,"num_object_clones":0,"num_object_copies":4041,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":10892,"num_write_kb":87934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10836,"ondisk_log_size":10836,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":406448,"kb_used_data":129808,"kb_used_omap":0,"kb_used_meta":276608,"kb_avail":377080912,"statfs":{"total":386547056640,"available":386130853888,"internally_reserved":0,"allocated":132923392,"data_stored":264762148,"data_compressed":13639026,"data_compressed_allocated":131354624,"data_compressed_original":264183808,"omap_allocated":0,"internal_metadata":283246592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11010048,"num_objects":167,"num_object_clones":0,"num_object_copies":501,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1344,"num_write_kb":10752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002486"},"pg_stats":[{"pgid":"6.f","version":"44'631","reported_seq":656,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:55.075080+0000","last_change":"2024-04-23T01:50:40.440701+0000","last_active":"2024-04-23T01:51:55.075080+0000","last_peered":"2024-04-23T01:51:55.075080+0000","last_clean":"2024-04-23T01:51:55.075080+0000","last_became_active":"2024-04-23T01:50:40.433832+0000","last_became_peered":"2024-04-23T01:50:40.433832+0000","last_unstale":"2024-04-23T01:51:55.075080+0000","last_undegraded":"2024-04-23T01:51:55.075080+0000","last_fullsized":"2024-04-23T01:51:55.075080+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":5056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"44'614","reported_seq":639,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:56.632597+0000","last_change":"2024-04-23T01:50:40.477278+0000","last_active":"2024-04-23T01:51:56.632597+0000","last_peered":"2024-04-23T01:51:56.632597+0000","last_clean":"2024-04-23T01:51:56.632597+0000","last_became_active":"2024-04-23T01:50:40.471274+0000","last_became_peered":"2024-04-23T01:50:40.471274+0000","last_unstale":"2024-04-23T01:51:56.632597+0000","last_undegraded":"2024-04-23T01:51:56.632597+0000","last_fullsized":"2024-04-23T01:51:56.632597+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5038080,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":615,"num_write_kb":4920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"44'671","reported_seq":696,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:56.367232+0000","last_change":"2024-04-23T01:50:40.446224+0000","last_active":"2024-04-23T01:51:56.367232+0000","last_peered":"2024-04-23T01:51:56.367232+0000","last_clean":"2024-04-23T01:51:56.367232+0000","last_became_active":"2024-04-23T01:50:40.437838+0000","last_became_peered":"2024-04-23T01:50:40.437838+0000","last_unstale":"2024-04-23T01:51:56.367232+0000","last_undegraded":"2024-04-23T01:51:56.367232+0000","last_fullsized":"2024-04-23T01:51:56.367232+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505024,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":672,"num_write_kb":5376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"44'635","reported_seq":660,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:56.598313+0000","last_change":"2024-04-23T01:50:40.517607+0000","last_active":"2024-04-23T01:51:56.598313+0000","last_peered":"2024-04-23T01:51:56.598313+0000","last_clean":"2024-04-23T01:51:56.598313+0000","last_became_active":"2024-04-23T01:50:40.511881+0000","last_became_peered":"2024-04-23T01:50:40.511881+0000","last_unstale":"2024-04-23T01:51:56.598313+0000","last_undegraded":"2024-04-23T01:51:56.598313+0000","last_fullsized":"2024-04-23T01:51:56.598313+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":635,"log_dups_size":0,"ondisk_log_size":635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5210112,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":636,"num_write_kb":5088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'847","reported_seq":872,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:55.061453+0000","last_change":"2024-04-23T01:50:40.432817+0000","last_active":"2024-04-23T01:51:55.061453+0000","last_peered":"2024-04-23T01:51:55.061453+0000","last_clean":"2024-04-23T01:51:55.061453+0000","last_became_active":"2024-04-23T01:50:40.421722+0000","last_became_peered":"2024-04-23T01:50:40.421722+0000","last_unstale":"2024-04-23T01:51:55.061453+0000","last_undegraded":"2024-04-23T01:51:55.061453+0000","last_fullsized":"2024-04-23T01:51:55.061453+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6946816,"num_objects":106,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":848,"num_write_kb":6784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"44'479","reported_seq":504,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:52.509782+0000","last_change":"2024-04-23T01:50:40.444510+0000","last_active":"2024-04-23T01:51:52.509782+0000","last_peered":"2024-04-23T01:51:52.509782+0000","last_clean":"2024-04-23T01:51:52.509782+0000","last_became_active":"2024-04-23T01:50:40.437088+0000","last_became_peered":"2024-04-23T01:50:40.437088+0000","last_unstale":"2024-04-23T01:51:52.509782+0000","last_undegraded":"2024-04-23T01:51:52.509782+0000","last_fullsized":"2024-04-23T01:51:52.509782+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":480,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"44'679","reported_seq":704,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:56.580913+0000","last_change":"2024-04-23T01:50:40.468123+0000","last_active":"2024-04-23T01:51:56.580913+0000","last_peered":"2024-04-23T01:51:56.580913+0000","last_clean":"2024-04-23T01:51:56.580913+0000","last_became_active":"2024-04-23T01:50:40.443838+0000","last_became_peered":"2024-04-23T01:50:40.443838+0000","last_unstale":"2024-04-23T01:51:56.580913+0000","last_undegraded":"2024-04-23T01:51:56.580913+0000","last_fullsized":"2024-04-23T01:51:56.580913+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5570560,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":680,"num_write_kb":5440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"44'591","reported_seq":616,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:55.667582+0000","last_change":"2024-04-23T01:50:40.451743+0000","last_active":"2024-04-23T01:51:55.667582+0000","last_peered":"2024-04-23T01:51:55.667582+0000","last_clean":"2024-04-23T01:51:55.667582+0000","last_became_active":"2024-04-23T01:50:40.447092+0000","last_became_peered":"2024-04-23T01:50:40.447092+0000","last_unstale":"2024-04-23T01:51:55.667582+0000","last_undegraded":"2024-04-23T01:51:55.667582+0000","last_fullsized":"2024-04-23T01:51:55.667582+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":592,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'591","reported_seq":616,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:55.950464+0000","last_change":"2024-04-23T01:50:40.477557+0000","last_active":"2024-04-23T01:51:55.950464+0000","last_peered":"2024-04-23T01:51:55.950464+0000","last_clean":"2024-04-23T01:51:55.950464+0000","last_became_active":"2024-04-23T01:50:40.472479+0000","last_became_peered":"2024-04-23T01:50:40.472479+0000","last_unstale":"2024-04-23T01:51:55.950464+0000","last_undegraded":"2024-04-23T01:51:55.950464+0000","last_fullsized":"2024-04-23T01:51:55.950464+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":592,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":227,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:13.373808+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:51:13.373808+0000","last_peered":"2024-04-23T01:51:13.373808+0000","last_clean":"2024-04-23T01:51:13.373808+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:51:13.373808+0000","last_undegraded":"2024-04-23T01:51:13.373808+0000","last_fullsized":"2024-04-23T01:51:13.373808+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":94,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:17.623030+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:51:17.623030+0000","last_peered":"2024-04-23T01:51:17.623030+0000","last_clean":"2024-04-23T01:51:17.623030+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:51:17.623030+0000","last_undegraded":"2024-04-23T01:51:17.623030+0000","last_fullsized":"2024-04-23T01:51:17.623030+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"44'639","reported_seq":664,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:54.395423+0000","last_change":"2024-04-23T01:50:40.431365+0000","last_active":"2024-04-23T01:51:54.395423+0000","last_peered":"2024-04-23T01:51:54.395423+0000","last_clean":"2024-04-23T01:51:54.395423+0000","last_became_active":"2024-04-23T01:50:40.426237+0000","last_became_peered":"2024-04-23T01:50:40.426237+0000","last_unstale":"2024-04-23T01:51:54.395423+0000","last_undegraded":"2024-04-23T01:51:54.395423+0000","last_fullsized":"2024-04-23T01:51:54.395423+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":640,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":92,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:16.753715+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:51:16.753715+0000","last_peered":"2024-04-23T01:51:16.753715+0000","last_clean":"2024-04-23T01:51:16.753715+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:51:16.753715+0000","last_undegraded":"2024-04-23T01:51:16.753715+0000","last_fullsized":"2024-04-23T01:51:16.753715+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"44'823","reported_seq":848,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:56.156209+0000","last_change":"2024-04-23T01:50:40.486856+0000","last_active":"2024-04-23T01:51:56.156209+0000","last_peered":"2024-04-23T01:51:56.156209+0000","last_clean":"2024-04-23T01:51:56.156209+0000","last_became_active":"2024-04-23T01:50:40.478453+0000","last_became_peered":"2024-04-23T01:50:40.478453+0000","last_unstale":"2024-04-23T01:51:56.156209+0000","last_undegraded":"2024-04-23T01:51:56.156209+0000","last_fullsized":"2024-04-23T01:51:56.156209+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":823,"log_dups_size":0,"ondisk_log_size":823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6750208,"num_objects":103,"num_object_clones":0,"num_object_copies":309,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":824,"num_write_kb":6592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":94,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:17.623032+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:51:17.623032+0000","last_peered":"2024-04-23T01:51:17.623032+0000","last_clean":"2024-04-23T01:51:17.623032+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:51:17.623032+0000","last_undegraded":"2024-04-23T01:51:17.623032+0000","last_fullsized":"2024-04-23T01:51:17.623032+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"44'623","reported_seq":648,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:56.491064+0000","last_change":"2024-04-23T01:50:40.462462+0000","last_active":"2024-04-23T01:51:56.491064+0000","last_peered":"2024-04-23T01:51:56.491064+0000","last_clean":"2024-04-23T01:51:56.491064+0000","last_became_active":"2024-04-23T01:50:40.439584+0000","last_became_peered":"2024-04-23T01:50:40.439584+0000","last_unstale":"2024-04-23T01:51:56.491064+0000","last_undegraded":"2024-04-23T01:51:56.491064+0000","last_fullsized":"2024-04-23T01:51:56.491064+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5111808,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":624,"num_write_kb":4992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":92,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:16.754133+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:51:16.754133+0000","last_peered":"2024-04-23T01:51:16.754133+0000","last_clean":"2024-04-23T01:51:16.754133+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:51:16.754133+0000","last_undegraded":"2024-04-23T01:51:16.754133+0000","last_fullsized":"2024-04-23T01:51:16.754133+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"44'719","reported_seq":744,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:55.262647+0000","last_change":"2024-04-23T01:50:40.466468+0000","last_active":"2024-04-23T01:51:55.262647+0000","last_peered":"2024-04-23T01:51:55.262647+0000","last_clean":"2024-04-23T01:51:55.262647+0000","last_became_active":"2024-04-23T01:50:40.452408+0000","last_became_peered":"2024-04-23T01:50:40.452408+0000","last_unstale":"2024-04-23T01:51:55.262647+0000","last_undegraded":"2024-04-23T01:51:55.262647+0000","last_fullsized":"2024-04-23T01:51:55.262647+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":720,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":98,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:16.748257+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:51:16.748257+0000","last_peered":"2024-04-23T01:51:16.748257+0000","last_clean":"2024-04-23T01:51:16.748257+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:51:16.748257+0000","last_undegraded":"2024-04-23T01:51:16.748257+0000","last_fullsized":"2024-04-23T01:51:16.748257+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"44'759","reported_seq":784,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:56.397267+0000","last_change":"2024-04-23T01:50:40.469709+0000","last_active":"2024-04-23T01:51:56.397267+0000","last_peered":"2024-04-23T01:51:56.397267+0000","last_clean":"2024-04-23T01:51:56.397267+0000","last_became_active":"2024-04-23T01:50:40.462062+0000","last_became_peered":"2024-04-23T01:50:40.462062+0000","last_unstale":"2024-04-23T01:51:56.397267+0000","last_undegraded":"2024-04-23T01:51:56.397267+0000","last_fullsized":"2024-04-23T01:51:56.397267+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6225920,"num_objects":95,"num_object_clones":0,"num_object_copies":285,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":760,"num_write_kb":6080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":94,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:13.380426+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:51:13.380426+0000","last_peered":"2024-04-23T01:51:13.380426+0000","last_clean":"2024-04-23T01:51:13.380426+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:51:13.380426+0000","last_undegraded":"2024-04-23T01:51:13.380426+0000","last_fullsized":"2024-04-23T01:51:13.380426+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"44'751","reported_seq":776,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:55.665753+0000","last_change":"2024-04-23T01:50:40.464715+0000","last_active":"2024-04-23T01:51:55.665753+0000","last_peered":"2024-04-23T01:51:55.665753+0000","last_clean":"2024-04-23T01:51:55.665753+0000","last_became_active":"2024-04-23T01:50:40.457999+0000","last_became_peered":"2024-04-23T01:50:40.457999+0000","last_unstale":"2024-04-23T01:51:55.665753+0000","last_undegraded":"2024-04-23T01:51:55.665753+0000","last_fullsized":"2024-04-23T01:51:55.665753+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":98,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:16.747086+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:51:16.747086+0000","last_peered":"2024-04-23T01:51:16.747086+0000","last_clean":"2024-04-23T01:51:16.747086+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:51:16.747086+0000","last_undegraded":"2024-04-23T01:51:16.747086+0000","last_fullsized":"2024-04-23T01:51:16.747086+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"44'687","reported_seq":712,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:54.229848+0000","last_change":"2024-04-23T01:50:40.447151+0000","last_active":"2024-04-23T01:51:54.229848+0000","last_peered":"2024-04-23T01:51:54.229848+0000","last_clean":"2024-04-23T01:51:54.229848+0000","last_became_active":"2024-04-23T01:50:40.441283+0000","last_became_peered":"2024-04-23T01:50:40.441283+0000","last_unstale":"2024-04-23T01:51:54.229848+0000","last_undegraded":"2024-04-23T01:51:54.229848+0000","last_fullsized":"2024-04-23T01:51:54.229848+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5636096,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":688,"num_write_kb":5504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":94,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:17.626169+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:51:17.626169+0000","last_peered":"2024-04-23T01:51:17.626169+0000","last_clean":"2024-04-23T01:51:17.626169+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:51:17.626169+0000","last_undegraded":"2024-04-23T01:51:17.626169+0000","last_fullsized":"2024-04-23T01:51:17.626169+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":88104960,"num_objects":1345,"num_object_clones":0,"num_object_copies":4035,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10755,"num_write_kb":86040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":132149248,"data_stored":264298496,"data_compressed":13643919,"data_compressed_allocated":132149248,"data_compressed_original":264298496,"omap_allocated":0,"internal_metadata":0},"log_size":10739,"ondisk_log_size":10739,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706073,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":101540,"kb_used_data":30300,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94270300,"statfs":{"total":96636764160,"available":96532787200,"internally_reserved":0,"allocated":31027200,"data_stored":61900113,"data_compressed":3188115,"data_compressed_allocated":30629888,"data_compressed_original":61751296,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706073,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":105736,"kb_used_data":30336,"kb_used_omap":0,"kb_used_meta":75392,"kb_avail":94266104,"statfs":{"total":96636764160,"available":96528490496,"internally_reserved":0,"allocated":31064064,"data_stored":61973841,"data_compressed":3191982,"data_compressed_allocated":30666752,"data_compressed_original":61825024,"omap_allocated":0,"internal_metadata":77201408},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706073,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":97480,"kb_used_data":34560,"kb_used_omap":0,"kb_used_meta":62912,"kb_avail":94274360,"statfs":{"total":96636764160,"available":96536944640,"internally_reserved":0,"allocated":35389440,"data_stored":70157105,"data_compressed":3614880,"data_compressed_allocated":35012608,"data_compressed_original":70025216,"omap_allocated":0,"internal_metadata":64421888},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706073,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":101692,"kb_used_data":34612,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94270148,"statfs":{"total":96636764160,"available":96532631552,"internally_reserved":0,"allocated":35442688,"data_stored":70731089,"data_compressed":3644049,"data_compressed_allocated":35045376,"data_compressed_original":70582272,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":35180544,"data_stored":70361088,"data_compressed":3632285,"data_compressed_allocated":35180544,"data_compressed_original":70361088,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":35311616,"data_stored":70623232,"data_compressed":3645759,"data_compressed_allocated":35311616,"data_compressed_original":70623232,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":30871552,"data_stored":61743104,"data_compressed":3187409,"data_compressed_allocated":30871552,"data_compressed_original":61743104,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":30785536,"data_stored":61571072,"data_compressed":3178466,"data_compressed_allocated":30785536,"data_compressed_original":61571072,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:51:59.981 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:52:00.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:00.094+0000 7fae10b1d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:00.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:00.100+0000 7fae10b1d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:00.358 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 79 15 11263 11248 1.11224 1.03906 0.142077 0.112288 2024-04-23T01:52:01.358 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:52:01.358321+0000 min lat: 0.0141099 max lat: 0.390523 avg lat: 0.11256 2024-04-23T01:52:01.358 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:52:01.358 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 80 16 11373 11357 1.10899 0.851562 0.1457 0.11256 2024-04-23T01:52:02.279 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:52:02.358 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 81 16 11498 11482 1.10735 0.976562 0.178766 0.112719 2024-04-23T01:52:02.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:02.414+0000 7f06f2d5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:02.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:02.422+0000 7f06f2d5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:03.358 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 82 16 11633 11617 1.10671 1.05469 0.227797 0.112826 2024-04-23T01:52:04.358 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 83 16 11765 11749 1.1058 1.03125 0.17196 0.112921 2024-04-23T01:52:04.559 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:52:04.672 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:52:04.672 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:52:04.672 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:52:04.672 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:52:04.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:04.717+0000 7f4e4c8ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:04.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:04.726+0000 7f4e4c8ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:04.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:04.790+0000 7f544a424640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:04.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:04.802+0000 7f544a424640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:04.831 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:52:04.831 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:52:04.831 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:52:04.831 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:52:04.831 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:52:04.831 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:52:04.832 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:52:04.832 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:52:04.832 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:52:04.832 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:52:04.832 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:52:04.832 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:52:04.834 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:52:04.834 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:52:04.834 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:52:04.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:04.952+0000 7f62ba75d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:04.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:04.964+0000 7f62ba75d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:04.994 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:52:04.994 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:52:04.994 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:52:04.994 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:52:04.994 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:52:04.994 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:52:04.994 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:52:04.994 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:52:04.994 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:52:04.994 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:52:04.995 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:52:04.995 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:52:04.997 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:52:04.998 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:52:04.998 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:52:05.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:05.182+0000 7fb13679d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:05.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:05.194+0000 7fb13679d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:05.251 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:52:05.251 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:52:05.251 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:52:05.251 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:52:05.251 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:52:05.251 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:52:05.251 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:52:05.251 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:52:05.251 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:52:05.251 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:52:05.251 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:52:05.251 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:52:05.254 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:52:05.254 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:52:05.255 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:52:05.358 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 84 16 11894 11878 1.10463 1.00781 0.0807696 0.113056 2024-04-23T01:52:05.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:05.391+0000 7f2a48b5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:05.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:05.404+0000 7f2a48b5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:05.440 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:52:05.441 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:52:05.441 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:52:05.441 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:52:05.441 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:52:05.441 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:52:05.441 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:52:05.441 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:52:05.441 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:52:05.441 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:52:05.441 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:52:05.441 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:52:05.443 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:52:05.443 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:52:06.358 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 85 16 12001 11985 1.10147 0.835938 0.133538 0.113365 2024-04-23T01:52:06.959 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:52:07.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:07.079+0000 7fcfa375d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:07.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:07.087+0000 7fcfa375d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:07.358 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 86 16 12121 12105 1.09956 0.9375 0.160158 0.113579 2024-04-23T01:52:08.358 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 87 16 12252 12236 1.09868 1.02344 0.0500511 0.113659 2024-04-23T01:52:09.306 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:52:09.358 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 88 16 12369 12353 1.09658 0.914062 0.236161 0.11387 2024-04-23T01:52:09.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:09.427+0000 7efed42ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:09.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:09.433+0000 7efed42ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:10.359 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 89 16 12490 12474 1.09488 0.945312 0.119506 0.114072 2024-04-23T01:52:10.444 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:52:10.445 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:52:10.445 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:52:10.445 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:52:10.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:10.561+0000 7fc1f9f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:10.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:10.575+0000 7fc1f9f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:10.605 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:52:10.605 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:52:10.605 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:52:10.605 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:52:10.605 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:52:10.605 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:52:10.605 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:52:10.605 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:52:10.605 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:52:10.605 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:52:10.605 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:52:10.605 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:52:10.609 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:52:10.609 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:52:10.609 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:52:10.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:10.726+0000 7f216da06640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:10.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:10.737+0000 7f216da06640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:10.786 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:52:10.786 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:52:10.786 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:52:10.786 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:52:10.786 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:52:10.786 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:52:10.786 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:52:10.786 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:52:10.786 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:52:10.786 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:52:10.786 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:52:10.787 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:52:10.790 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:52:10.791 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:52:10.791 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:52:10.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:10.930+0000 7fd20e4ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:10.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:10.940+0000 7fd20e4ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:10.978 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:52:10.979 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:52:10.979 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:52:10.979 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:52:10.979 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:52:10.979 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:52:10.979 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:52:10.979 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:52:10.979 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:52:10.980 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:52:10.980 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:52:10.980 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:52:10.981 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:52:10.982 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:52:10.982 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:52:11.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:11.129+0000 7fd9a9b5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:11.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:11.140+0000 7fd9a9b5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:11.206 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:52:11.206 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:52:11.206 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:52:11.206 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:52:11.206 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:52:11.206 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:52:11.206 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:52:11.206 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:52:11.206 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:52:11.206 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:52:11.206 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:52:11.206 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:52:11.209 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:52:11.209 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:52:11.359 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 90 14 12614 12600 1.09365 0.984375 0.0388696 0.114212 2024-04-23T01:52:11.585 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:52:11.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:11.691+0000 7f3e9539d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:11.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:11.697+0000 7f3e9539d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:12.359 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Total time run: 90.1086 2024-04-23T01:52:12.359 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Total writes made: 12614 2024-04-23T01:52:12.359 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Write size: 8192 2024-04-23T01:52:12.360 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Object size: 65536 2024-04-23T01:52:12.360 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Bandwidth (MB/sec): 1.09365 2024-04-23T01:52:12.360 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Stddev Bandwidth: 0.103968 2024-04-23T01:52:12.360 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Max bandwidth (MB/sec): 1.35156 2024-04-23T01:52:12.360 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Min bandwidth (MB/sec): 0.835938 2024-04-23T01:52:12.360 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Average IOPS: 139 2024-04-23T01:52:12.360 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Stddev IOPS: 13.3079 2024-04-23T01:52:12.360 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Max IOPS: 173 2024-04-23T01:52:12.360 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Min IOPS: 107 2024-04-23T01:52:12.360 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Average Latency(s): 0.114223 2024-04-23T01:52:12.360 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Stddev Latency(s): 0.0576758 2024-04-23T01:52:12.360 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Max latency(s): 0.390523 2024-04-23T01:52:12.360 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Min latency(s): 0.0141099 2024-04-23T01:52:12.382 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Cleaning up (deleting benchmark objects) 2024-04-23T01:52:13.790 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:52:13.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:13.929+0000 7f2b1415d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:13.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:13.937+0000 7f2b1415d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:16.210 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:52:16.210 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:52:16.210 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:52:16.211 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:52:16.254 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:52:16.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:16.333+0000 7f3a0b75d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:16.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:16.348+0000 7f3a0b75d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:16.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:16.388+0000 7f7895d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:16.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:16.397+0000 7f7895d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:16.401 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:52:16.401 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:52:16.401 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:52:16.401 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:52:16.401 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:52:16.401 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:52:16.401 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:52:16.402 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:52:16.402 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:52:16.402 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:52:16.402 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:52:16.402 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:52:16.404 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:52:16.404 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:52:16.405 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:52:16.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:16.537+0000 7f3aa9a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:16.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:16.548+0000 7f3aa9a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:16.700 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:52:16.700 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:52:16.700 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:52:16.700 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:52:16.700 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:52:16.700 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:52:16.701 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:52:16.701 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:52:16.701 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:52:16.701 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:52:16.701 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:52:16.701 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:52:16.704 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:52:16.704 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:52:16.705 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:52:16.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:16.843+0000 7f602bead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:16.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:16.856+0000 7f602bead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:17.100 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:52:17.144 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:52:17.145 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:52:17.145 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:52:17.145 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:52:17.145 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:52:17.145 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:52:17.145 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:52:17.145 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:52:17.145 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:52:17.145 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:52:17.145 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:52:17.146 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:52:17.147 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:52:17.147 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:52:17.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:17.275+0000 7fbf2346d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:17.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:17.288+0000 7fbf2346d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:17.450 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:52:17.450 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:52:17.450 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:52:17.450 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:52:17.450 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:52:17.450 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:52:17.450 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:52:17.450 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:52:17.451 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:52:17.451 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:52:17.451 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:52:17.451 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:52:17.453 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:52:17.453 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:52:18.782 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:52:18.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:18.913+0000 7f7a5b59d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:18.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:18.920+0000 7f7a5b59d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:21.287 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:52:21.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:21.419+0000 7fcd6119d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:21.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:21.424+0000 7fcd6119d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:22.073 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Removed 1577 objects 2024-04-23T01:52:22.073 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Clean up completed and total clean up time :9.69043 2024-04-23T01:52:22.454 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:52:22.454 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:52:22.454 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:52:22.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:22.556+0000 7f403b79d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:22.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:22.563+0000 7f403b79d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:22.751 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:52:22.751 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:52:22.764 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":255,"stamp":"2024-04-23T01:52:21.632200+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":57524792,"num_objects":872,"num_object_clones":0,"num_object_copies":2616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":13460,"num_write_kb":102807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13404,"ondisk_log_size":13404,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":381672,"kb_used_data":92520,"kb_used_omap":0,"kb_used_meta":289088,"kb_avail":377105688,"statfs":{"total":386547056640,"available":386156224512,"internally_reserved":0,"allocated":94740480,"data_stored":188371820,"data_compressed":9697878,"data_compressed_allocated":93159424,"data_compressed_original":187793408,"omap_allocated":0,"internal_metadata":296026112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-41557992,"num_objects":-634,"num_object_clones":0,"num_object_copies":-1902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1300,"num_write_kb":4729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002196"},"pg_stats":[{"pgid":"6.f","version":"44'804","reported_seq":829,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:52:16.616432+0000","last_change":"2024-04-23T01:50:40.440701+0000","last_active":"2024-04-23T01:52:16.616432+0000","last_peered":"2024-04-23T01:52:16.616432+0000","last_clean":"2024-04-23T01:52:16.616432+0000","last_became_active":"2024-04-23T01:50:40.433832+0000","last_became_peered":"2024-04-23T01:50:40.433832+0000","last_unstale":"2024-04-23T01:52:16.616432+0000","last_undegraded":"2024-04-23T01:52:16.616432+0000","last_fullsized":"2024-04-23T01:52:16.616432+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3276800,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":805,"num_write_kb":6080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"44'792","reported_seq":817,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:52:16.572146+0000","last_change":"2024-04-23T01:50:40.477278+0000","last_active":"2024-04-23T01:52:16.572146+0000","last_peered":"2024-04-23T01:52:16.572146+0000","last_clean":"2024-04-23T01:52:16.572146+0000","last_became_active":"2024-04-23T01:50:40.471274+0000","last_became_peered":"2024-04-23T01:50:40.471274+0000","last_unstale":"2024-04-23T01:52:16.572146+0000","last_undegraded":"2024-04-23T01:52:16.572146+0000","last_fullsized":"2024-04-23T01:52:16.572146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":792,"log_dups_size":0,"ondisk_log_size":792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3473408,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":793,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"44'822","reported_seq":848,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:52:16.510365+0000","last_change":"2024-04-23T01:50:40.446224+0000","last_active":"2024-04-23T01:52:16.510365+0000","last_peered":"2024-04-23T01:52:16.510365+0000","last_clean":"2024-04-23T01:52:16.510365+0000","last_became_active":"2024-04-23T01:50:40.437838+0000","last_became_peered":"2024-04-23T01:50:40.437838+0000","last_unstale":"2024-04-23T01:52:16.510365+0000","last_undegraded":"2024-04-23T01:52:16.510365+0000","last_fullsized":"2024-04-23T01:52:16.510365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":822,"log_dups_size":0,"ondisk_log_size":822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342360,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":823,"num_write_kb":6209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"44'764","reported_seq":789,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:52:16.552934+0000","last_change":"2024-04-23T01:50:40.517607+0000","last_active":"2024-04-23T01:52:16.552934+0000","last_peered":"2024-04-23T01:52:16.552934+0000","last_clean":"2024-04-23T01:52:16.552934+0000","last_became_active":"2024-04-23T01:50:40.511881+0000","last_became_peered":"2024-04-23T01:50:40.511881+0000","last_unstale":"2024-04-23T01:52:16.552934+0000","last_undegraded":"2024-04-23T01:52:16.552934+0000","last_fullsized":"2024-04-23T01:52:16.552934+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":764,"log_dups_size":0,"ondisk_log_size":764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4128768,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":765,"num_write_kb":5888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'1037","reported_seq":1062,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:52:16.624147+0000","last_change":"2024-04-23T01:50:40.432817+0000","last_active":"2024-04-23T01:52:16.624147+0000","last_peered":"2024-04-23T01:52:16.624147+0000","last_clean":"2024-04-23T01:52:16.624147+0000","last_became_active":"2024-04-23T01:50:40.421722+0000","last_became_peered":"2024-04-23T01:50:40.421722+0000","last_unstale":"2024-04-23T01:52:16.624147+0000","last_undegraded":"2024-04-23T01:52:16.624147+0000","last_fullsized":"2024-04-23T01:52:16.624147+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":1037,"log_dups_size":0,"ondisk_log_size":1037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4521984,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1038,"num_write_kb":7872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"44'616","reported_seq":641,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:52:16.398874+0000","last_change":"2024-04-23T01:50:40.444510+0000","last_active":"2024-04-23T01:52:16.398874+0000","last_peered":"2024-04-23T01:52:16.398874+0000","last_clean":"2024-04-23T01:52:16.398874+0000","last_became_active":"2024-04-23T01:50:40.437088+0000","last_became_peered":"2024-04-23T01:50:40.437088+0000","last_unstale":"2024-04-23T01:52:16.398874+0000","last_undegraded":"2024-04-23T01:52:16.398874+0000","last_fullsized":"2024-04-23T01:52:16.398874+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":617,"num_write_kb":4672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"44'889","reported_seq":914,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:52:16.431753+0000","last_change":"2024-04-23T01:50:40.468123+0000","last_active":"2024-04-23T01:52:16.431753+0000","last_peered":"2024-04-23T01:52:16.431753+0000","last_clean":"2024-04-23T01:52:16.431753+0000","last_became_active":"2024-04-23T01:50:40.443838+0000","last_became_peered":"2024-04-23T01:50:40.443838+0000","last_unstale":"2024-04-23T01:52:16.431753+0000","last_undegraded":"2024-04-23T01:52:16.431753+0000","last_fullsized":"2024-04-23T01:52:16.431753+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":889,"log_dups_size":0,"ondisk_log_size":889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3604480,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":890,"num_write_kb":6720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"44'735","reported_seq":760,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:52:16.190056+0000","last_change":"2024-04-23T01:50:40.451743+0000","last_active":"2024-04-23T01:52:16.190056+0000","last_peered":"2024-04-23T01:52:16.190056+0000","last_clean":"2024-04-23T01:52:16.190056+0000","last_became_active":"2024-04-23T01:50:40.447092+0000","last_became_peered":"2024-04-23T01:50:40.447092+0000","last_unstale":"2024-04-23T01:52:16.190056+0000","last_undegraded":"2024-04-23T01:52:16.190056+0000","last_fullsized":"2024-04-23T01:52:16.190056+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2932736,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":736,"num_write_kb":5552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'707","reported_seq":732,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:52:16.584444+0000","last_change":"2024-04-23T01:50:40.477557+0000","last_active":"2024-04-23T01:52:16.584444+0000","last_peered":"2024-04-23T01:52:16.584444+0000","last_clean":"2024-04-23T01:52:16.584444+0000","last_became_active":"2024-04-23T01:50:40.472479+0000","last_became_peered":"2024-04-23T01:50:40.472479+0000","last_unstale":"2024-04-23T01:52:16.584444+0000","last_undegraded":"2024-04-23T01:52:16.584444+0000","last_fullsized":"2024-04-23T01:52:16.584444+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":708,"num_write_kb":5376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":227,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:13.373808+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:51:13.373808+0000","last_peered":"2024-04-23T01:51:13.373808+0000","last_clean":"2024-04-23T01:51:13.373808+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:51:13.373808+0000","last_undegraded":"2024-04-23T01:51:13.373808+0000","last_fullsized":"2024-04-23T01:51:13.373808+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":94,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:17.623030+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:51:17.623030+0000","last_peered":"2024-04-23T01:51:17.623030+0000","last_clean":"2024-04-23T01:51:17.623030+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:51:17.623030+0000","last_undegraded":"2024-04-23T01:51:17.623030+0000","last_fullsized":"2024-04-23T01:51:17.623030+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"44'768","reported_seq":793,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:52:16.374880+0000","last_change":"2024-04-23T01:50:40.431365+0000","last_active":"2024-04-23T01:52:16.374880+0000","last_peered":"2024-04-23T01:52:16.374880+0000","last_clean":"2024-04-23T01:52:16.374880+0000","last_became_active":"2024-04-23T01:50:40.426237+0000","last_became_peered":"2024-04-23T01:50:40.426237+0000","last_unstale":"2024-04-23T01:52:16.374880+0000","last_undegraded":"2024-04-23T01:52:16.374880+0000","last_fullsized":"2024-04-23T01:52:16.374880+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":768,"log_dups_size":0,"ondisk_log_size":768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3276800,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":769,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":92,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:16.753715+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:51:16.753715+0000","last_peered":"2024-04-23T01:51:16.753715+0000","last_clean":"2024-04-23T01:51:16.753715+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:51:16.753715+0000","last_undegraded":"2024-04-23T01:51:16.753715+0000","last_fullsized":"2024-04-23T01:51:16.753715+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"44'1041","reported_seq":1066,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:52:16.569885+0000","last_change":"2024-04-23T01:50:40.486856+0000","last_active":"2024-04-23T01:52:16.569885+0000","last_peered":"2024-04-23T01:52:16.569885+0000","last_clean":"2024-04-23T01:52:16.569885+0000","last_became_active":"2024-04-23T01:50:40.478453+0000","last_became_peered":"2024-04-23T01:50:40.478453+0000","last_unstale":"2024-04-23T01:52:16.569885+0000","last_undegraded":"2024-04-23T01:52:16.569885+0000","last_fullsized":"2024-04-23T01:52:16.569885+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":1041,"log_dups_size":0,"ondisk_log_size":1041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1042,"num_write_kb":7936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":94,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:17.623032+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:51:17.623032+0000","last_peered":"2024-04-23T01:51:17.623032+0000","last_clean":"2024-04-23T01:51:17.623032+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:51:17.623032+0000","last_undegraded":"2024-04-23T01:51:17.623032+0000","last_fullsized":"2024-04-23T01:51:17.623032+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"44'767","reported_seq":792,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:52:16.216790+0000","last_change":"2024-04-23T01:50:40.462462+0000","last_active":"2024-04-23T01:52:16.216790+0000","last_peered":"2024-04-23T01:52:16.216790+0000","last_clean":"2024-04-23T01:52:16.216790+0000","last_became_active":"2024-04-23T01:50:40.439584+0000","last_became_peered":"2024-04-23T01:50:40.439584+0000","last_unstale":"2024-04-23T01:52:16.216790+0000","last_undegraded":"2024-04-23T01:52:16.216790+0000","last_fullsized":"2024-04-23T01:52:16.216790+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342336,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":768,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":92,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:16.754133+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:51:16.754133+0000","last_peered":"2024-04-23T01:51:16.754133+0000","last_clean":"2024-04-23T01:51:16.754133+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:51:16.754133+0000","last_undegraded":"2024-04-23T01:51:16.754133+0000","last_fullsized":"2024-04-23T01:51:16.754133+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"44'915","reported_seq":940,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:52:16.440067+0000","last_change":"2024-04-23T01:50:40.466468+0000","last_active":"2024-04-23T01:52:16.440067+0000","last_peered":"2024-04-23T01:52:16.440067+0000","last_clean":"2024-04-23T01:52:16.440067+0000","last_became_active":"2024-04-23T01:50:40.452408+0000","last_became_peered":"2024-04-23T01:50:40.452408+0000","last_unstale":"2024-04-23T01:52:16.440067+0000","last_undegraded":"2024-04-23T01:52:16.440067+0000","last_fullsized":"2024-04-23T01:52:16.440067+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":915,"log_dups_size":0,"ondisk_log_size":915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":916,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":98,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:16.748257+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:51:16.748257+0000","last_peered":"2024-04-23T01:51:16.748257+0000","last_clean":"2024-04-23T01:51:16.748257+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:51:16.748257+0000","last_undegraded":"2024-04-23T01:51:16.748257+0000","last_fullsized":"2024-04-23T01:51:16.748257+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"44'917","reported_seq":942,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:52:16.367483+0000","last_change":"2024-04-23T01:50:40.469709+0000","last_active":"2024-04-23T01:52:16.367483+0000","last_peered":"2024-04-23T01:52:16.367483+0000","last_clean":"2024-04-23T01:52:16.367483+0000","last_became_active":"2024-04-23T01:50:40.462062+0000","last_became_peered":"2024-04-23T01:50:40.462062+0000","last_unstale":"2024-04-23T01:52:16.367483+0000","last_undegraded":"2024-04-23T01:52:16.367483+0000","last_fullsized":"2024-04-23T01:52:16.367483+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":917,"log_dups_size":0,"ondisk_log_size":917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3538944,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":918,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":94,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:13.380426+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:51:13.380426+0000","last_peered":"2024-04-23T01:51:13.380426+0000","last_clean":"2024-04-23T01:51:13.380426+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:51:13.380426+0000","last_undegraded":"2024-04-23T01:51:13.380426+0000","last_fullsized":"2024-04-23T01:51:13.380426+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"44'883","reported_seq":908,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:52:16.316312+0000","last_change":"2024-04-23T01:50:40.464715+0000","last_active":"2024-04-23T01:52:16.316312+0000","last_peered":"2024-04-23T01:52:16.316312+0000","last_clean":"2024-04-23T01:52:16.316312+0000","last_became_active":"2024-04-23T01:50:40.457999+0000","last_became_peered":"2024-04-23T01:50:40.457999+0000","last_unstale":"2024-04-23T01:52:16.316312+0000","last_undegraded":"2024-04-23T01:52:16.316312+0000","last_fullsized":"2024-04-23T01:52:16.316312+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3997696,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":884,"num_write_kb":6720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":98,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:16.747086+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:51:16.747086+0000","last_peered":"2024-04-23T01:51:16.747086+0000","last_clean":"2024-04-23T01:51:16.747086+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:51:16.747086+0000","last_undegraded":"2024-04-23T01:51:16.747086+0000","last_fullsized":"2024-04-23T01:51:16.747086+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"44'850","reported_seq":875,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:52:16.421536+0000","last_change":"2024-04-23T01:50:40.447151+0000","last_active":"2024-04-23T01:52:16.421536+0000","last_peered":"2024-04-23T01:52:16.421536+0000","last_clean":"2024-04-23T01:52:16.421536+0000","last_became_active":"2024-04-23T01:50:40.441283+0000","last_became_peered":"2024-04-23T01:50:40.441283+0000","last_unstale":"2024-04-23T01:52:16.421536+0000","last_undegraded":"2024-04-23T01:52:16.421536+0000","last_fullsized":"2024-04-23T01:52:16.421536+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:50:39.257496+0000","last_clean_scrub_stamp":"2024-04-23T01:50:39.257496+0000","objects_scrubbed":0,"log_size":850,"log_dups_size":0,"ondisk_log_size":850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3211264,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":851,"num_write_kb":6400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":94,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T01:51:17.626169+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:51:17.626169+0000","last_peered":"2024-04-23T01:51:17.626169+0000","last_clean":"2024-04-23T01:51:17.626169+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:51:17.626169+0000","last_undegraded":"2024-04-23T01:51:17.626169+0000","last_fullsized":"2024-04-23T01:51:17.626169+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":56934424,"num_objects":870,"num_object_clones":0,"num_object_copies":2610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13323,"num_write_kb":100913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":85676032,"data_stored":171327560,"data_compressed":8846619,"data_compressed_allocated":85663744,"data_compressed_original":171327488,"omap_allocated":0,"internal_metadata":0},"log_size":13307,"ondisk_log_size":13307,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706093,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":97456,"kb_used_data":22024,"kb_used_omap":0,"kb_used_meta":75392,"kb_avail":94274384,"statfs":{"total":96636764160,"available":96536969216,"internally_reserved":0,"allocated":22552576,"data_stored":44942697,"data_compressed":2313306,"data_compressed_allocated":22151168,"data_compressed_original":44793856,"omap_allocated":0,"internal_metadata":77201408},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706093,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":89260,"kb_used_data":22180,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94282580,"statfs":{"total":96636764160,"available":96545361920,"internally_reserved":0,"allocated":22712320,"data_stored":45270353,"data_compressed":2330226,"data_compressed_allocated":22315008,"data_compressed_original":45121536,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706093,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":95708,"kb_used_data":24468,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94276132,"statfs":{"total":96636764160,"available":96538759168,"internally_reserved":0,"allocated":25055232,"data_stored":49480521,"data_compressed":2548151,"data_compressed_allocated":24674304,"data_compressed_original":49348608,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706093,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":99248,"kb_used_data":23848,"kb_used_omap":0,"kb_used_meta":75392,"kb_avail":94272592,"statfs":{"total":96636764160,"available":96535134208,"internally_reserved":0,"allocated":24420352,"data_stored":48678249,"data_compressed":2506195,"data_compressed_allocated":24018944,"data_compressed_original":48529408,"omap_allocated":0,"internal_metadata":77201408},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22441984,"data_stored":44875800,"data_compressed":2317194,"data_compressed_allocated":22437888,"data_compressed_original":44875776,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22876160,"data_stored":45744152,"data_compressed":2362031,"data_compressed_allocated":22872064,"data_compressed_original":45744128,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20275200,"data_stored":40550400,"data_compressed":2093849,"data_compressed_allocated":20275200,"data_compressed_original":40550400,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20082688,"data_stored":40157208,"data_compressed":2073545,"data_compressed_allocated":20078592,"data_compressed_original":40157184,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:52:23.962 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:52:24.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:24.073+0000 7fbda359d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:24.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:24.079+0000 7fbda359d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:25.015 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_3 2024-04-23T01:52:25.015 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_3 unique_pool_3 --yes-i-really-really-mean-it 2024-04-23T01:52:25.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:25.119+0000 7f12ca0ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:25.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:25.131+0000 7f12ca0ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:26.177 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:52:26.245 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'unique_pool_3' removed 2024-04-23T01:52:26.263 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-04-23T01:52:26.264 INFO:tasks.radosbench:Beginning radosbench... 2024-04-23T01:52:26.264 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_4 2024-04-23T01:52:26.265 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_4 16 2024-04-23T01:52:26.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:26.305+0000 7f07a0e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:26.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:26.315+0000 7f07a0e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:26.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:26.381+0000 7ff4e106d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:26.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:26.392+0000 7ff4e106d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:27.254 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'unique_pool_4' created 2024-04-23T01:52:27.274 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_4 rados --yes-i-really-mean-it || true 2024-04-23T01:52:27.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:27.417+0000 7fe043d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:27.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:27.434+0000 7fe043d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:27.767 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:52:27.767 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:52:27.767 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:52:27.767 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:52:27.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:27.868+0000 7fb21d8ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:27.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:27.877+0000 7fb21d8ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:27.892 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:52:27.892 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:52:27.892 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:52:27.892 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:52:27.892 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:52:27.892 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:52:27.892 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:52:27.892 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:52:27.893 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:52:27.893 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:52:27.893 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:52:27.893 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:52:27.895 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:52:27.895 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:52:27.896 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:52:28.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:27.997+0000 7f3457b9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:28.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:28.006+0000 7f3457b9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:28.021 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:52:28.021 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:52:28.021 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:52:28.021 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:52:28.021 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:52:28.021 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:52:28.021 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:52:28.021 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:52:28.021 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:52:28.021 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:52:28.021 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:52:28.021 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:52:28.023 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:52:28.024 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:52:28.024 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:52:28.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:28.125+0000 7feca0aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:28.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:28.133+0000 7feca0aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:28.150 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:52:28.150 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:52:28.150 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:52:28.150 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:52:28.150 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:52:28.150 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:52:28.150 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:52:28.150 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:52:28.150 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:52:28.151 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:52:28.151 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:52:28.151 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:52:28.152 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:52:28.153 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:52:28.153 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:52:28.258 INFO:teuthology.orchestra.run.smithi090.stderr:enabled application 'rados' on pool 'unique_pool_4' 2024-04-23T01:52:28.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:28.256+0000 7f8e1f71d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:28.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:28.271+0000 7f8e1f71d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:28.330 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:52:28.330 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:52:28.330 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:52:28.330 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:52:28.330 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:52:28.330 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:52:28.330 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:52:28.330 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:52:28.330 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:52:28.330 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:52:28.330 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:52:28.331 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:52:28.334 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:52:28.334 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:52:28.460 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:52:28.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:28.577+0000 7f676ae6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:28.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:28.583+0000 7f676ae6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:29.275 DEBUG:teuthology.orchestra.run.smithi090:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 8192 --object-size 65536 -p unique_pool_4 bench 90 write' 2024-04-23T01:52:29.278 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-04-23T01:52:29.279 INFO:teuthology.orchestra.run:waiting for 3000 2024-04-23T01:52:29.339 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:hints = 1 2024-04-23T01:52:29.339 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Maintaining 16 concurrent writes of 8192 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-04-23T01:52:29.339 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Object prefix: benchmark_data_smithi090_46174 2024-04-23T01:52:29.339 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:52:29.339 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 0 0 0 0 0 0 - 0 2024-04-23T01:52:30.339 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 1 16 155 139 1.08587 1.08594 0.200799 0.107164 2024-04-23T01:52:30.677 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:52:30.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:30.801+0000 7f6d6122d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:30.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:30.809+0000 7f6d6122d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:31.339 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 2 16 313 297 1.16007 1.23438 0.0589649 0.103895 2024-04-23T01:52:32.339 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 3 16 485 469 1.22126 1.34375 0.0981827 0.100347 2024-04-23T01:52:33.059 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:52:33.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:33.169+0000 7fe9b499d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:33.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:33.177+0000 7fe9b499d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:33.335 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:52:33.335 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:52:33.335 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:52:33.339 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 4 16 632 616 1.20303 1.14844 0.165217 0.102252 2024-04-23T01:52:33.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:33.464+0000 7fc231d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:33.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:33.474+0000 7fc231d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:33.736 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:52:33.737 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:52:33.759 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2024-04-23T01:52:33.634663+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3539488,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":497,"num_write_kb":4774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":441,"ondisk_log_size":441,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":277544,"kb_used_data":5060,"kb_used_omap":0,"kb_used_meta":272448,"kb_avail":377209816,"statfs":{"total":386547056640,"available":386262851584,"internally_reserved":0,"allocated":5181440,"data_stored":9039312,"data_compressed":433881,"data_compressed_allocated":3465216,"data_compressed_original":8404992,"omap_allocated":0,"internal_metadata":278986752},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2949120,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":360,"num_write_kb":2880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.377414"},"pg_stats":[{"pgid":"7.f","version":"48'15","reported_seq":28,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:31.630007+0000","last_change":"2024-04-23T01:52:28.525229+0000","last_active":"2024-04-23T01:52:31.630007+0000","last_peered":"2024-04-23T01:52:31.630007+0000","last_clean":"2024-04-23T01:52:31.630007+0000","last_became_active":"2024-04-23T01:52:28.518971+0000","last_became_peered":"2024-04-23T01:52:28.518971+0000","last_unstale":"2024-04-23T01:52:31.630007+0000","last_undegraded":"2024-04-23T01:52:31.630007+0000","last_fullsized":"2024-04-23T01:52:31.630007+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"48'15","reported_seq":28,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:30.419019+0000","last_change":"2024-04-23T01:52:28.530871+0000","last_active":"2024-04-23T01:52:30.419019+0000","last_peered":"2024-04-23T01:52:30.419019+0000","last_clean":"2024-04-23T01:52:30.419019+0000","last_became_active":"2024-04-23T01:52:28.521696+0000","last_became_peered":"2024-04-23T01:52:28.521696+0000","last_unstale":"2024-04-23T01:52:30.419019+0000","last_undegraded":"2024-04-23T01:52:30.419019+0000","last_fullsized":"2024-04-23T01:52:30.419019+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"48'26","reported_seq":39,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:31.663465+0000","last_change":"2024-04-23T01:52:28.407738+0000","last_active":"2024-04-23T01:52:31.663465+0000","last_peered":"2024-04-23T01:52:31.663465+0000","last_clean":"2024-04-23T01:52:31.663465+0000","last_became_active":"2024-04-23T01:52:28.400739+0000","last_became_peered":"2024-04-23T01:52:28.400739+0000","last_unstale":"2024-04-23T01:52:31.663465+0000","last_undegraded":"2024-04-23T01:52:31.663465+0000","last_fullsized":"2024-04-23T01:52:31.663465+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":221184,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"48'23","reported_seq":36,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:30.563097+0000","last_change":"2024-04-23T01:52:28.435954+0000","last_active":"2024-04-23T01:52:30.563097+0000","last_peered":"2024-04-23T01:52:30.563097+0000","last_clean":"2024-04-23T01:52:30.563097+0000","last_became_active":"2024-04-23T01:52:28.430557+0000","last_became_peered":"2024-04-23T01:52:28.430557+0000","last_unstale":"2024-04-23T01:52:30.563097+0000","last_undegraded":"2024-04-23T01:52:30.563097+0000","last_fullsized":"2024-04-23T01:52:30.563097+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"48'24","reported_seq":37,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:31.007903+0000","last_change":"2024-04-23T01:52:28.452034+0000","last_active":"2024-04-23T01:52:31.007903+0000","last_peered":"2024-04-23T01:52:31.007903+0000","last_clean":"2024-04-23T01:52:31.007903+0000","last_became_active":"2024-04-23T01:52:28.430826+0000","last_became_peered":"2024-04-23T01:52:28.430826+0000","last_unstale":"2024-04-23T01:52:31.007903+0000","last_undegraded":"2024-04-23T01:52:31.007903+0000","last_fullsized":"2024-04-23T01:52:31.007903+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":204800,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"48'38","reported_seq":51,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:31.672100+0000","last_change":"2024-04-23T01:52:28.484337+0000","last_active":"2024-04-23T01:52:31.672100+0000","last_peered":"2024-04-23T01:52:31.672100+0000","last_clean":"2024-04-23T01:52:31.672100+0000","last_became_active":"2024-04-23T01:52:28.476566+0000","last_became_peered":"2024-04-23T01:52:28.476566+0000","last_unstale":"2024-04-23T01:52:31.672100+0000","last_undegraded":"2024-04-23T01:52:31.672100+0000","last_fullsized":"2024-04-23T01:52:31.672100+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319488,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"48'47","reported_seq":60,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:31.482573+0000","last_change":"2024-04-23T01:52:28.418531+0000","last_active":"2024-04-23T01:52:31.482573+0000","last_peered":"2024-04-23T01:52:31.482573+0000","last_clean":"2024-04-23T01:52:31.482573+0000","last_became_active":"2024-04-23T01:52:28.412725+0000","last_became_peered":"2024-04-23T01:52:28.412725+0000","last_unstale":"2024-04-23T01:52:31.482573+0000","last_undegraded":"2024-04-23T01:52:31.482573+0000","last_fullsized":"2024-04-23T01:52:31.482573+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"48'15","reported_seq":28,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:31.463756+0000","last_change":"2024-04-23T01:52:28.498385+0000","last_active":"2024-04-23T01:52:31.463756+0000","last_peered":"2024-04-23T01:52:31.463756+0000","last_clean":"2024-04-23T01:52:31.463756+0000","last_became_active":"2024-04-23T01:52:28.487852+0000","last_became_peered":"2024-04-23T01:52:28.487852+0000","last_unstale":"2024-04-23T01:52:31.463756+0000","last_undegraded":"2024-04-23T01:52:31.463756+0000","last_fullsized":"2024-04-23T01:52:31.463756+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"48'15","reported_seq":28,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.760368+0000","last_change":"2024-04-23T01:52:28.536598+0000","last_active":"2024-04-23T01:52:29.760368+0000","last_peered":"2024-04-23T01:52:29.760368+0000","last_clean":"2024-04-23T01:52:29.760368+0000","last_became_active":"2024-04-23T01:52:28.530283+0000","last_became_peered":"2024-04-23T01:52:28.530283+0000","last_unstale":"2024-04-23T01:52:29.760368+0000","last_undegraded":"2024-04-23T01:52:29.760368+0000","last_fullsized":"2024-04-23T01:52:29.760368+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":237,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.359805+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:52:29.359805+0000","last_peered":"2024-04-23T01:52:29.359805+0000","last_clean":"2024-04-23T01:52:29.359805+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:52:29.359805+0000","last_undegraded":"2024-04-23T01:52:29.359805+0000","last_fullsized":"2024-04-23T01:52:29.359805+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.374210+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:52:29.374210+0000","last_peered":"2024-04-23T01:52:29.374210+0000","last_clean":"2024-04-23T01:52:29.374210+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:52:29.374210+0000","last_undegraded":"2024-04-23T01:52:29.374210+0000","last_fullsized":"2024-04-23T01:52:29.374210+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"48'23","reported_seq":36,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:31.377049+0000","last_change":"2024-04-23T01:52:28.481837+0000","last_active":"2024-04-23T01:52:31.377049+0000","last_peered":"2024-04-23T01:52:31.377049+0000","last_clean":"2024-04-23T01:52:31.377049+0000","last_became_active":"2024-04-23T01:52:28.472032+0000","last_became_peered":"2024-04-23T01:52:28.472032+0000","last_unstale":"2024-04-23T01:52:31.377049+0000","last_undegraded":"2024-04-23T01:52:31.377049+0000","last_fullsized":"2024-04-23T01:52:31.377049+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":102,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.628985+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:52:29.628985+0000","last_peered":"2024-04-23T01:52:29.628985+0000","last_clean":"2024-04-23T01:52:29.628985+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:52:29.628985+0000","last_undegraded":"2024-04-23T01:52:29.628985+0000","last_fullsized":"2024-04-23T01:52:29.628985+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"48'31","reported_seq":44,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:31.324682+0000","last_change":"2024-04-23T01:52:28.529295+0000","last_active":"2024-04-23T01:52:31.324682+0000","last_peered":"2024-04-23T01:52:31.324682+0000","last_clean":"2024-04-23T01:52:31.324682+0000","last_became_active":"2024-04-23T01:52:28.523577+0000","last_became_peered":"2024-04-23T01:52:28.523577+0000","last_unstale":"2024-04-23T01:52:31.324682+0000","last_undegraded":"2024-04-23T01:52:31.324682+0000","last_fullsized":"2024-04-23T01:52:31.324682+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.368290+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:52:29.368290+0000","last_peered":"2024-04-23T01:52:29.368290+0000","last_clean":"2024-04-23T01:52:29.368290+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:52:29.368290+0000","last_undegraded":"2024-04-23T01:52:29.368290+0000","last_fullsized":"2024-04-23T01:52:29.368290+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"48'23","reported_seq":36,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:31.258087+0000","last_change":"2024-04-23T01:52:28.456738+0000","last_active":"2024-04-23T01:52:31.258087+0000","last_peered":"2024-04-23T01:52:31.258087+0000","last_clean":"2024-04-23T01:52:31.258087+0000","last_became_active":"2024-04-23T01:52:28.432791+0000","last_became_peered":"2024-04-23T01:52:28.432791+0000","last_unstale":"2024-04-23T01:52:31.258087+0000","last_undegraded":"2024-04-23T01:52:31.258087+0000","last_fullsized":"2024-04-23T01:52:31.258087+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":102,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.623888+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:52:29.623888+0000","last_peered":"2024-04-23T01:52:29.623888+0000","last_clean":"2024-04-23T01:52:29.623888+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:52:29.623888+0000","last_undegraded":"2024-04-23T01:52:29.623888+0000","last_fullsized":"2024-04-23T01:52:29.623888+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"48'20","reported_seq":33,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:31.602118+0000","last_change":"2024-04-23T01:52:28.427753+0000","last_active":"2024-04-23T01:52:31.602118+0000","last_peered":"2024-04-23T01:52:31.602118+0000","last_clean":"2024-04-23T01:52:31.602118+0000","last_became_active":"2024-04-23T01:52:28.423538+0000","last_became_peered":"2024-04-23T01:52:28.423538+0000","last_unstale":"2024-04-23T01:52:31.602118+0000","last_undegraded":"2024-04-23T01:52:31.602118+0000","last_fullsized":"2024-04-23T01:52:31.602118+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":172032,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":108,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.278611+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:52:29.278611+0000","last_peered":"2024-04-23T01:52:29.278611+0000","last_clean":"2024-04-23T01:52:29.278611+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:52:29.278611+0000","last_undegraded":"2024-04-23T01:52:29.278611+0000","last_fullsized":"2024-04-23T01:52:29.278611+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"48'15","reported_seq":28,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:31.522600+0000","last_change":"2024-04-23T01:52:28.503432+0000","last_active":"2024-04-23T01:52:31.522600+0000","last_peered":"2024-04-23T01:52:31.522600+0000","last_clean":"2024-04-23T01:52:31.522600+0000","last_became_active":"2024-04-23T01:52:28.498953+0000","last_became_peered":"2024-04-23T01:52:28.498953+0000","last_unstale":"2024-04-23T01:52:31.522600+0000","last_undegraded":"2024-04-23T01:52:31.522600+0000","last_fullsized":"2024-04-23T01:52:31.522600+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":104,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.389394+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:52:29.389394+0000","last_peered":"2024-04-23T01:52:29.389394+0000","last_clean":"2024-04-23T01:52:29.389394+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:52:29.389394+0000","last_undegraded":"2024-04-23T01:52:29.389394+0000","last_fullsized":"2024-04-23T01:52:29.389394+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"48'7","reported_seq":20,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:30.705817+0000","last_change":"2024-04-23T01:52:28.486992+0000","last_active":"2024-04-23T01:52:30.705817+0000","last_peered":"2024-04-23T01:52:30.705817+0000","last_clean":"2024-04-23T01:52:30.705817+0000","last_became_active":"2024-04-23T01:52:28.482639+0000","last_became_peered":"2024-04-23T01:52:28.482639+0000","last_unstale":"2024-04-23T01:52:30.705817+0000","last_undegraded":"2024-04-23T01:52:30.705817+0000","last_fullsized":"2024-04-23T01:52:30.705817+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65536,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":108,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.297789+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:52:29.297789+0000","last_peered":"2024-04-23T01:52:29.297789+0000","last_clean":"2024-04-23T01:52:29.297789+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:52:29.297789+0000","last_undegraded":"2024-04-23T01:52:29.297789+0000","last_fullsized":"2024-04-23T01:52:29.297789+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"48'7","reported_seq":20,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:30.619711+0000","last_change":"2024-04-23T01:52:28.453502+0000","last_active":"2024-04-23T01:52:30.619711+0000","last_peered":"2024-04-23T01:52:30.619711+0000","last_clean":"2024-04-23T01:52:30.619711+0000","last_became_active":"2024-04-23T01:52:28.448199+0000","last_became_peered":"2024-04-23T01:52:28.448199+0000","last_unstale":"2024-04-23T01:52:30.619711+0000","last_undegraded":"2024-04-23T01:52:30.619711+0000","last_fullsized":"2024-04-23T01:52:30.619711+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65536,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.366780+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:52:29.366780+0000","last_peered":"2024-04-23T01:52:29.366780+0000","last_clean":"2024-04-23T01:52:29.366780+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:52:29.366780+0000","last_undegraded":"2024-04-23T01:52:29.366780+0000","last_fullsized":"2024-04-23T01:52:29.366780+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":2949120,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":360,"num_write_kb":2880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4399104,"data_stored":8798208,"data_compressed":452898,"data_compressed_allocated":4399104,"data_compressed_original":8798208,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706108,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":64368,"kb_used_data":1448,"kb_used_omap":0,"kb_used_meta":62912,"kb_avail":94307472,"statfs":{"total":96636764160,"available":96570851328,"internally_reserved":0,"allocated":1482752,"data_stored":2751484,"data_compressed":133655,"data_compressed_allocated":1048576,"data_compressed_original":2588672,"omap_allocated":0,"internal_metadata":64421888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706108,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":72400,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94299440,"statfs":{"total":96636764160,"available":96562626560,"internally_reserved":0,"allocated":1187840,"data_stored":2161660,"data_compressed":103329,"data_compressed_allocated":753664,"data_compressed_original":1998848,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706108,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":76664,"kb_used_data":1264,"kb_used_omap":0,"kb_used_meta":75392,"kb_avail":94295176,"statfs":{"total":96636764160,"available":96558260224,"internally_reserved":0,"allocated":1294336,"data_stored":1907164,"data_compressed":90636,"data_compressed_allocated":880640,"data_compressed_original":1761280,"omap_allocated":0,"internal_metadata":77201408},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706108,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":64112,"kb_used_data":1188,"kb_used_omap":0,"kb_used_meta":62912,"kb_avail":94307728,"statfs":{"total":96636764160,"available":96571113472,"internally_reserved":0,"allocated":1216512,"data_stored":2219004,"data_compressed":106261,"data_compressed_allocated":782336,"data_compressed_original":2056192,"omap_allocated":0,"internal_metadata":64421888},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":884736,"data_stored":1769472,"data_compressed":91078,"data_compressed_allocated":884736,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1138688,"data_stored":2277376,"data_compressed":117222,"data_compressed_allocated":1138688,"data_compressed_original":2277376,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":962560,"data_stored":1925120,"data_compressed":99118,"data_compressed_allocated":962560,"data_compressed_original":1925120,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1413120,"data_stored":2826240,"data_compressed":145480,"data_compressed_allocated":1413120,"data_compressed_original":2826240,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:52:34.339 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 5 16 789 773 1.20771 1.22656 0.182583 0.102277 2024-04-23T01:52:35.339 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 6 16 943 927 1.20693 1.20312 0.0245683 0.102485 2024-04-23T01:52:35.358 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:52:35.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:35.483+0000 7f105f4ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:35.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:35.489+0000 7f105f4ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:36.339 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 7 16 1084 1068 1.19187 1.10156 0.131532 0.103574 2024-04-23T01:52:37.340 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 8 16 1230 1214 1.18545 1.14062 0.0291518 0.10454 2024-04-23T01:52:37.700 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:52:37.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:37.823+0000 7f642331d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:37.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:37.831+0000 7f642331d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:38.340 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 9 16 1352 1336 1.15963 0.953125 0.246885 0.105882 2024-04-23T01:52:38.761 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:52:38.761 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:52:38.762 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:52:38.762 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:52:38.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:38.901+0000 7f549c6ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:38.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:38.913+0000 7f549c6ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:38.940 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:52:38.940 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:52:38.940 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:52:38.940 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:52:38.940 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:52:38.940 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:52:38.941 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:52:38.941 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:52:38.941 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:52:38.941 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:52:38.941 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:52:38.941 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:52:38.944 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:52:38.944 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:52:38.944 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:52:39.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:39.093+0000 7f8a32aed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:39.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:39.105+0000 7f8a32aed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:39.152 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:52:39.152 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:52:39.152 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:52:39.152 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:52:39.152 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:52:39.152 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:52:39.152 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:52:39.152 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:52:39.152 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:52:39.152 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:52:39.152 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:52:39.153 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:52:39.155 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:52:39.156 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:52:39.156 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:52:39.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:39.287+0000 7f754f8ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:39.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:39.298+0000 7f754f8ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:39.313 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:52:39.314 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:52:39.314 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:52:39.314 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:52:39.314 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:52:39.314 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:52:39.314 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:52:39.314 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:52:39.314 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:52:39.314 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:52:39.315 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:52:39.315 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:52:39.317 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:52:39.317 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:52:39.317 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:52:39.339 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 10 16 1483 1467 1.146 1.02344 0.0971191 0.108107 2024-04-23T01:52:39.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:39.438+0000 7f320d617640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:39.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:39.448+0000 7f320d617640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:39.487 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:52:39.488 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:52:39.488 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:52:39.488 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:52:39.488 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:52:39.488 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:52:39.488 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:52:39.488 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:52:39.488 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:52:39.488 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:52:39.488 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:52:39.488 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:52:39.490 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:52:39.491 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:52:40.088 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:52:40.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:40.202+0000 7f9b138ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:40.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:40.209+0000 7f9b138ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:40.340 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 11 16 1630 1614 1.14622 1.14844 0.153806 0.108408 2024-04-23T01:52:41.340 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 12 16 1779 1763 1.1477 1.16406 0.147987 0.108098 2024-04-23T01:52:42.340 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 13 16 1915 1899 1.14113 1.0625 0.144652 0.108732 2024-04-23T01:52:42.410 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:52:42.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:42.557+0000 7fadb546d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:42.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:42.563+0000 7fadb546d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:43.340 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 14 16 2059 2043 1.13998 1.125 0.0461915 0.109088 2024-04-23T01:52:44.340 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 15 16 2224 2208 1.14991 1.28906 0.161502 0.108367 2024-04-23T01:52:44.492 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:52:44.492 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:52:44.492 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:52:44.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:44.606+0000 7f435c617640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:44.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:44.616+0000 7f435c617640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:44.739 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:52:44.844 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:52:44.845 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:52:44.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:44.853+0000 7f12dc42d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:44.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:44.859+0000 7f12dc42d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:44.861 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":270,"stamp":"2024-04-23T01:52:43.636561+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":15475232,"num_objects":230,"num_object_clones":0,"num_object_copies":690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":1954,"num_write_kb":16430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1898,"ondisk_log_size":1898,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":335508,"kb_used_data":22772,"kb_used_omap":0,"kb_used_meta":312704,"kb_avail":377151852,"statfs":{"total":386547056640,"available":386203496448,"internally_reserved":0,"allocated":23318528,"data_stored":45313488,"data_compressed":2304741,"data_compressed_allocated":21602304,"data_compressed_original":44679168,"omap_allocated":0,"internal_metadata":320208896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14884864,"num_objects":228,"num_object_clones":0,"num_object_copies":684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1817,"num_write_kb":14536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002255"},"pg_stats":[{"pgid":"7.f","version":"48'103","reported_seq":116,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:40.650270+0000","last_change":"2024-04-23T01:52:28.525229+0000","last_active":"2024-04-23T01:52:40.650270+0000","last_peered":"2024-04-23T01:52:40.650270+0000","last_clean":"2024-04-23T01:52:40.650270+0000","last_became_active":"2024-04-23T01:52:28.518971+0000","last_became_peered":"2024-04-23T01:52:28.518971+0000","last_unstale":"2024-04-23T01:52:40.650270+0000","last_undegraded":"2024-04-23T01:52:40.650270+0000","last_fullsized":"2024-04-23T01:52:40.650270+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"48'103","reported_seq":116,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:41.013334+0000","last_change":"2024-04-23T01:52:28.530871+0000","last_active":"2024-04-23T01:52:41.013334+0000","last_peered":"2024-04-23T01:52:41.013334+0000","last_clean":"2024-04-23T01:52:41.013334+0000","last_became_active":"2024-04-23T01:52:28.521696+0000","last_became_peered":"2024-04-23T01:52:28.521696+0000","last_unstale":"2024-04-23T01:52:41.013334+0000","last_undegraded":"2024-04-23T01:52:41.013334+0000","last_fullsized":"2024-04-23T01:52:41.013334+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"48'127","reported_seq":140,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:41.534680+0000","last_change":"2024-04-23T01:52:28.407738+0000","last_active":"2024-04-23T01:52:41.534680+0000","last_peered":"2024-04-23T01:52:41.534680+0000","last_clean":"2024-04-23T01:52:41.534680+0000","last_became_active":"2024-04-23T01:52:28.400739+0000","last_became_peered":"2024-04-23T01:52:28.400739+0000","last_unstale":"2024-04-23T01:52:41.534680+0000","last_undegraded":"2024-04-23T01:52:41.534680+0000","last_fullsized":"2024-04-23T01:52:41.534680+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"48'41","reported_seq":54,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:41.672556+0000","last_change":"2024-04-23T01:52:28.435954+0000","last_active":"2024-04-23T01:52:41.672556+0000","last_peered":"2024-04-23T01:52:41.672556+0000","last_clean":"2024-04-23T01:52:41.672556+0000","last_became_active":"2024-04-23T01:52:28.430557+0000","last_became_peered":"2024-04-23T01:52:28.430557+0000","last_unstale":"2024-04-23T01:52:41.672556+0000","last_undegraded":"2024-04-23T01:52:41.672556+0000","last_fullsized":"2024-04-23T01:52:41.672556+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":344064,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"48'111","reported_seq":124,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:40.544281+0000","last_change":"2024-04-23T01:52:28.452034+0000","last_active":"2024-04-23T01:52:40.544281+0000","last_peered":"2024-04-23T01:52:40.544281+0000","last_clean":"2024-04-23T01:52:40.544281+0000","last_became_active":"2024-04-23T01:52:28.430826+0000","last_became_peered":"2024-04-23T01:52:28.430826+0000","last_unstale":"2024-04-23T01:52:40.544281+0000","last_undegraded":"2024-04-23T01:52:40.544281+0000","last_fullsized":"2024-04-23T01:52:40.544281+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"48'183","reported_seq":196,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:41.635922+0000","last_change":"2024-04-23T01:52:28.484337+0000","last_active":"2024-04-23T01:52:41.635922+0000","last_peered":"2024-04-23T01:52:41.635922+0000","last_clean":"2024-04-23T01:52:41.635922+0000","last_became_active":"2024-04-23T01:52:28.476566+0000","last_became_peered":"2024-04-23T01:52:28.476566+0000","last_unstale":"2024-04-23T01:52:41.635922+0000","last_undegraded":"2024-04-23T01:52:41.635922+0000","last_fullsized":"2024-04-23T01:52:41.635922+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"48'134","reported_seq":147,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:41.686868+0000","last_change":"2024-04-23T01:52:28.418531+0000","last_active":"2024-04-23T01:52:41.686868+0000","last_peered":"2024-04-23T01:52:41.686868+0000","last_clean":"2024-04-23T01:52:41.686868+0000","last_became_active":"2024-04-23T01:52:28.412725+0000","last_became_peered":"2024-04-23T01:52:28.412725+0000","last_unstale":"2024-04-23T01:52:41.686868+0000","last_undegraded":"2024-04-23T01:52:41.686868+0000","last_fullsized":"2024-04-23T01:52:41.686868+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1105920,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":1080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"48'119","reported_seq":132,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:40.506535+0000","last_change":"2024-04-23T01:52:28.498385+0000","last_active":"2024-04-23T01:52:40.506535+0000","last_peered":"2024-04-23T01:52:40.506535+0000","last_clean":"2024-04-23T01:52:40.506535+0000","last_became_active":"2024-04-23T01:52:28.487852+0000","last_became_peered":"2024-04-23T01:52:28.487852+0000","last_unstale":"2024-04-23T01:52:40.506535+0000","last_undegraded":"2024-04-23T01:52:40.506535+0000","last_fullsized":"2024-04-23T01:52:40.506535+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"48'103","reported_seq":116,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:40.223792+0000","last_change":"2024-04-23T01:52:28.536598+0000","last_active":"2024-04-23T01:52:40.223792+0000","last_peered":"2024-04-23T01:52:40.223792+0000","last_clean":"2024-04-23T01:52:40.223792+0000","last_became_active":"2024-04-23T01:52:28.530283+0000","last_became_peered":"2024-04-23T01:52:28.530283+0000","last_unstale":"2024-04-23T01:52:40.223792+0000","last_undegraded":"2024-04-23T01:52:40.223792+0000","last_fullsized":"2024-04-23T01:52:40.223792+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":237,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.359805+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:52:29.359805+0000","last_peered":"2024-04-23T01:52:29.359805+0000","last_clean":"2024-04-23T01:52:29.359805+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:52:29.359805+0000","last_undegraded":"2024-04-23T01:52:29.359805+0000","last_fullsized":"2024-04-23T01:52:29.359805+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.374210+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:52:29.374210+0000","last_peered":"2024-04-23T01:52:29.374210+0000","last_clean":"2024-04-23T01:52:29.374210+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:52:29.374210+0000","last_undegraded":"2024-04-23T01:52:29.374210+0000","last_fullsized":"2024-04-23T01:52:29.374210+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"48'151","reported_seq":164,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:40.793514+0000","last_change":"2024-04-23T01:52:28.481837+0000","last_active":"2024-04-23T01:52:40.793514+0000","last_peered":"2024-04-23T01:52:40.793514+0000","last_clean":"2024-04-23T01:52:40.793514+0000","last_became_active":"2024-04-23T01:52:28.472032+0000","last_became_peered":"2024-04-23T01:52:28.472032+0000","last_unstale":"2024-04-23T01:52:40.793514+0000","last_undegraded":"2024-04-23T01:52:40.793514+0000","last_fullsized":"2024-04-23T01:52:40.793514+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":102,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.628985+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:52:29.628985+0000","last_peered":"2024-04-23T01:52:29.628985+0000","last_clean":"2024-04-23T01:52:29.628985+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:52:29.628985+0000","last_undegraded":"2024-04-23T01:52:29.628985+0000","last_fullsized":"2024-04-23T01:52:29.628985+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"48'175","reported_seq":188,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:41.553010+0000","last_change":"2024-04-23T01:52:28.529295+0000","last_active":"2024-04-23T01:52:41.553010+0000","last_peered":"2024-04-23T01:52:41.553010+0000","last_clean":"2024-04-23T01:52:41.553010+0000","last_became_active":"2024-04-23T01:52:28.523577+0000","last_became_peered":"2024-04-23T01:52:28.523577+0000","last_unstale":"2024-04-23T01:52:41.553010+0000","last_undegraded":"2024-04-23T01:52:41.553010+0000","last_fullsized":"2024-04-23T01:52:41.553010+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.368290+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:52:29.368290+0000","last_peered":"2024-04-23T01:52:29.368290+0000","last_clean":"2024-04-23T01:52:29.368290+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:52:29.368290+0000","last_undegraded":"2024-04-23T01:52:29.368290+0000","last_fullsized":"2024-04-23T01:52:29.368290+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"48'87","reported_seq":100,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:40.650659+0000","last_change":"2024-04-23T01:52:28.456738+0000","last_active":"2024-04-23T01:52:40.650659+0000","last_peered":"2024-04-23T01:52:40.650659+0000","last_clean":"2024-04-23T01:52:40.650659+0000","last_became_active":"2024-04-23T01:52:28.432791+0000","last_became_peered":"2024-04-23T01:52:28.432791+0000","last_unstale":"2024-04-23T01:52:40.650659+0000","last_undegraded":"2024-04-23T01:52:40.650659+0000","last_fullsized":"2024-04-23T01:52:40.650659+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":720896,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":102,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.623888+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:52:29.623888+0000","last_peered":"2024-04-23T01:52:29.623888+0000","last_clean":"2024-04-23T01:52:29.623888+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:52:29.623888+0000","last_undegraded":"2024-04-23T01:52:29.623888+0000","last_fullsized":"2024-04-23T01:52:29.623888+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"48'143","reported_seq":156,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:41.600897+0000","last_change":"2024-04-23T01:52:28.427753+0000","last_active":"2024-04-23T01:52:41.600897+0000","last_peered":"2024-04-23T01:52:41.600897+0000","last_clean":"2024-04-23T01:52:41.600897+0000","last_became_active":"2024-04-23T01:52:28.423538+0000","last_became_peered":"2024-04-23T01:52:28.423538+0000","last_unstale":"2024-04-23T01:52:41.600897+0000","last_undegraded":"2024-04-23T01:52:41.600897+0000","last_fullsized":"2024-04-23T01:52:41.600897+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":108,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.278611+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:52:29.278611+0000","last_peered":"2024-04-23T01:52:29.278611+0000","last_clean":"2024-04-23T01:52:29.278611+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:52:29.278611+0000","last_undegraded":"2024-04-23T01:52:29.278611+0000","last_fullsized":"2024-04-23T01:52:29.278611+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"48'71","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:40.994093+0000","last_change":"2024-04-23T01:52:28.503432+0000","last_active":"2024-04-23T01:52:40.994093+0000","last_peered":"2024-04-23T01:52:40.994093+0000","last_clean":"2024-04-23T01:52:40.994093+0000","last_became_active":"2024-04-23T01:52:28.498953+0000","last_became_peered":"2024-04-23T01:52:28.498953+0000","last_unstale":"2024-04-23T01:52:40.994093+0000","last_undegraded":"2024-04-23T01:52:40.994093+0000","last_fullsized":"2024-04-23T01:52:40.994093+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":104,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.389394+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:52:29.389394+0000","last_peered":"2024-04-23T01:52:29.389394+0000","last_clean":"2024-04-23T01:52:29.389394+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:52:29.389394+0000","last_undegraded":"2024-04-23T01:52:29.389394+0000","last_fullsized":"2024-04-23T01:52:29.389394+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"48'79","reported_seq":92,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:41.099074+0000","last_change":"2024-04-23T01:52:28.486992+0000","last_active":"2024-04-23T01:52:41.099074+0000","last_peered":"2024-04-23T01:52:41.099074+0000","last_clean":"2024-04-23T01:52:41.099074+0000","last_became_active":"2024-04-23T01:52:28.482639+0000","last_became_peered":"2024-04-23T01:52:28.482639+0000","last_unstale":"2024-04-23T01:52:41.099074+0000","last_undegraded":"2024-04-23T01:52:41.099074+0000","last_fullsized":"2024-04-23T01:52:41.099074+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":108,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.297789+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:52:29.297789+0000","last_peered":"2024-04-23T01:52:29.297789+0000","last_clean":"2024-04-23T01:52:29.297789+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:52:29.297789+0000","last_undegraded":"2024-04-23T01:52:29.297789+0000","last_fullsized":"2024-04-23T01:52:29.297789+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"48'71","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:40.122244+0000","last_change":"2024-04-23T01:52:28.453502+0000","last_active":"2024-04-23T01:52:40.122244+0000","last_peered":"2024-04-23T01:52:40.122244+0000","last_clean":"2024-04-23T01:52:40.122244+0000","last_became_active":"2024-04-23T01:52:28.448199+0000","last_became_peered":"2024-04-23T01:52:28.448199+0000","last_unstale":"2024-04-23T01:52:40.122244+0000","last_undegraded":"2024-04-23T01:52:40.122244+0000","last_fullsized":"2024-04-23T01:52:40.122244+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.366780+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:52:29.366780+0000","last_peered":"2024-04-23T01:52:29.366780+0000","last_clean":"2024-04-23T01:52:29.366780+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:52:29.366780+0000","last_undegraded":"2024-04-23T01:52:29.366780+0000","last_fullsized":"2024-04-23T01:52:29.366780+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":14884864,"num_objects":228,"num_object_clones":0,"num_object_copies":684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1817,"num_write_kb":14536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":22310912,"data_stored":44621824,"data_compressed":2300751,"data_compressed_allocated":22310912,"data_compressed_original":44621824,"omap_allocated":0,"internal_metadata":0},"log_size":1801,"ondisk_log_size":1801,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706118,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":96292,"kb_used_data":7004,"kb_used_omap":0,"kb_used_meta":89280,"kb_avail":94275548,"statfs":{"total":96636764160,"available":96538161152,"internally_reserved":0,"allocated":7172096,"data_stored":14130172,"data_compressed":720488,"data_compressed_allocated":6737920,"data_compressed_original":13967360,"omap_allocated":0,"internal_metadata":91422720},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706118,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":79904,"kb_used_data":4504,"kb_used_omap":0,"kb_used_meta":75392,"kb_avail":94291936,"statfs":{"total":96636764160,"available":96554942464,"internally_reserved":0,"allocated":4612096,"data_stored":9010172,"data_compressed":456555,"data_compressed_allocated":4177920,"data_compressed_original":8847360,"omap_allocated":0,"internal_metadata":77201408},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706118,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":68688,"kb_used_data":5768,"kb_used_omap":0,"kb_used_meta":62912,"kb_avail":94303152,"statfs":{"total":96636764160,"available":96566427648,"internally_reserved":0,"allocated":5906432,"data_stored":11131356,"data_compressed":566389,"data_compressed_allocated":5492736,"data_compressed_original":10985472,"omap_allocated":0,"internal_metadata":64421888},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706118,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":90624,"kb_used_data":5496,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94281216,"statfs":{"total":96636764160,"available":96543965184,"internally_reserved":0,"allocated":5627904,"data_stored":11041788,"data_compressed":561309,"data_compressed_allocated":5193728,"data_compressed_original":10878976,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5267456,"data_stored":10534912,"data_compressed":543200,"data_compressed_allocated":5267456,"data_compressed_original":10534912,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5619712,"data_stored":11239424,"data_compressed":579502,"data_compressed_allocated":5619712,"data_compressed_original":11239424,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4415488,"data_stored":8830976,"data_compressed":455366,"data_compressed_allocated":4415488,"data_compressed_original":8830976,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7008256,"data_stored":14016512,"data_compressed":722683,"data_compressed_allocated":7008256,"data_compressed_original":14016512,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:52:45.340 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 16 16 2370 2354 1.14932 1.14062 0.130705 0.108406 2024-04-23T01:52:46.341 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 17 16 2522 2506 1.15156 1.1875 0.123088 0.108194 2024-04-23T01:52:47.138 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:52:47.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:47.255+0000 7f599466d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:47.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:47.262+0000 7f599466d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:47.340 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 18 16 2670 2654 1.15182 1.15625 0.0378156 0.108044 2024-04-23T01:52:48.340 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 19 16 2820 2804 1.15287 1.17188 0.0902387 0.108091 2024-04-23T01:52:49.341 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:52:49.341024+0000 min lat: 0.0158935 max lat: 0.310257 avg lat: 0.107504 2024-04-23T01:52:49.341 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:52:49.341 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 20 16 2984 2968 1.15928 1.28125 0.0355992 0.107504 2024-04-23T01:52:49.484 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:52:49.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:49.609+0000 7fe0a462d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:49.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:49.615+0000 7fe0a462d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:49.863 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:52:49.863 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:52:49.863 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:52:49.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:49.983+0000 7fb4dbbff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:49.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:49.991+0000 7fb4dbbff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:50.313 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:52:50.314 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:52:50.333 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":273,"stamp":"2024-04-23T01:52:49.637723+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":21561888,"num_objects":324,"num_object_clones":0,"num_object_copies":972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":2697,"num_write_kb":22374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2641,"ondisk_log_size":2641,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":348544,"kb_used_data":31648,"kb_used_omap":0,"kb_used_meta":316864,"kb_avail":377138816,"statfs":{"total":386547056640,"available":386190147584,"internally_reserved":0,"allocated":32407552,"data_stored":63491536,"data_compressed":3243378,"data_compressed_allocated":30691328,"data_compressed_original":62857216,"omap_allocated":0,"internal_metadata":324468736},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11878400,"num_objects":181,"num_object_clones":0,"num_object_copies":543,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1450,"num_write_kb":11600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002337"},"pg_stats":[{"pgid":"7.f","version":"48'136","reported_seq":149,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:46.369208+0000","last_change":"2024-04-23T01:52:28.525229+0000","last_active":"2024-04-23T01:52:46.369208+0000","last_peered":"2024-04-23T01:52:46.369208+0000","last_clean":"2024-04-23T01:52:46.369208+0000","last_became_active":"2024-04-23T01:52:28.518971+0000","last_became_peered":"2024-04-23T01:52:28.518971+0000","last_unstale":"2024-04-23T01:52:46.369208+0000","last_undegraded":"2024-04-23T01:52:46.369208+0000","last_fullsized":"2024-04-23T01:52:46.369208+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122304,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":137,"num_write_kb":1096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"48'159","reported_seq":172,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:45.847587+0000","last_change":"2024-04-23T01:52:28.530871+0000","last_active":"2024-04-23T01:52:45.847587+0000","last_peered":"2024-04-23T01:52:45.847587+0000","last_clean":"2024-04-23T01:52:45.847587+0000","last_became_active":"2024-04-23T01:52:28.521696+0000","last_became_peered":"2024-04-23T01:52:28.521696+0000","last_unstale":"2024-04-23T01:52:45.847587+0000","last_undegraded":"2024-04-23T01:52:45.847587+0000","last_fullsized":"2024-04-23T01:52:45.847587+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310720,"num_objects":20,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":160,"num_write_kb":1280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"48'151","reported_seq":164,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:46.597470+0000","last_change":"2024-04-23T01:52:28.407738+0000","last_active":"2024-04-23T01:52:46.597470+0000","last_peered":"2024-04-23T01:52:46.597470+0000","last_clean":"2024-04-23T01:52:46.597470+0000","last_became_active":"2024-04-23T01:52:28.400739+0000","last_became_peered":"2024-04-23T01:52:28.400739+0000","last_unstale":"2024-04-23T01:52:46.597470+0000","last_undegraded":"2024-04-23T01:52:46.597470+0000","last_fullsized":"2024-04-23T01:52:46.597470+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"48'103","reported_seq":116,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:46.252852+0000","last_change":"2024-04-23T01:52:28.435954+0000","last_active":"2024-04-23T01:52:46.252852+0000","last_peered":"2024-04-23T01:52:46.252852+0000","last_clean":"2024-04-23T01:52:46.252852+0000","last_became_active":"2024-04-23T01:52:28.430557+0000","last_became_peered":"2024-04-23T01:52:28.430557+0000","last_unstale":"2024-04-23T01:52:46.252852+0000","last_undegraded":"2024-04-23T01:52:46.252852+0000","last_fullsized":"2024-04-23T01:52:46.252852+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"48'167","reported_seq":180,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:46.385012+0000","last_change":"2024-04-23T01:52:28.452034+0000","last_active":"2024-04-23T01:52:46.385012+0000","last_peered":"2024-04-23T01:52:46.385012+0000","last_clean":"2024-04-23T01:52:46.385012+0000","last_became_active":"2024-04-23T01:52:28.430826+0000","last_became_peered":"2024-04-23T01:52:28.430826+0000","last_unstale":"2024-04-23T01:52:46.385012+0000","last_undegraded":"2024-04-23T01:52:46.385012+0000","last_fullsized":"2024-04-23T01:52:46.385012+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":1344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"48'250","reported_seq":263,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:46.678096+0000","last_change":"2024-04-23T01:52:28.484337+0000","last_active":"2024-04-23T01:52:46.678096+0000","last_peered":"2024-04-23T01:52:46.678096+0000","last_clean":"2024-04-23T01:52:46.678096+0000","last_became_active":"2024-04-23T01:52:28.476566+0000","last_became_peered":"2024-04-23T01:52:28.476566+0000","last_unstale":"2024-04-23T01:52:46.678096+0000","last_undegraded":"2024-04-23T01:52:46.678096+0000","last_fullsized":"2024-04-23T01:52:46.678096+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2056192,"num_objects":32,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":251,"num_write_kb":2008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"48'183","reported_seq":196,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:45.279321+0000","last_change":"2024-04-23T01:52:28.418531+0000","last_active":"2024-04-23T01:52:45.279321+0000","last_peered":"2024-04-23T01:52:45.279321+0000","last_clean":"2024-04-23T01:52:45.279321+0000","last_became_active":"2024-04-23T01:52:28.412725+0000","last_became_peered":"2024-04-23T01:52:28.412725+0000","last_unstale":"2024-04-23T01:52:45.279321+0000","last_undegraded":"2024-04-23T01:52:45.279321+0000","last_fullsized":"2024-04-23T01:52:45.279321+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"48'183","reported_seq":196,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:46.436011+0000","last_change":"2024-04-23T01:52:28.498385+0000","last_active":"2024-04-23T01:52:46.436011+0000","last_peered":"2024-04-23T01:52:46.436011+0000","last_clean":"2024-04-23T01:52:46.436011+0000","last_became_active":"2024-04-23T01:52:28.487852+0000","last_became_peered":"2024-04-23T01:52:28.487852+0000","last_unstale":"2024-04-23T01:52:46.436011+0000","last_undegraded":"2024-04-23T01:52:46.436011+0000","last_fullsized":"2024-04-23T01:52:46.436011+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"48'135","reported_seq":148,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:45.372695+0000","last_change":"2024-04-23T01:52:28.536598+0000","last_active":"2024-04-23T01:52:45.372695+0000","last_peered":"2024-04-23T01:52:45.372695+0000","last_clean":"2024-04-23T01:52:45.372695+0000","last_became_active":"2024-04-23T01:52:28.530283+0000","last_became_peered":"2024-04-23T01:52:28.530283+0000","last_unstale":"2024-04-23T01:52:45.372695+0000","last_undegraded":"2024-04-23T01:52:45.372695+0000","last_fullsized":"2024-04-23T01:52:45.372695+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":136,"num_write_kb":1088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":237,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.359805+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:52:29.359805+0000","last_peered":"2024-04-23T01:52:29.359805+0000","last_clean":"2024-04-23T01:52:29.359805+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:52:29.359805+0000","last_undegraded":"2024-04-23T01:52:29.359805+0000","last_fullsized":"2024-04-23T01:52:29.359805+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.374210+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:52:29.374210+0000","last_peered":"2024-04-23T01:52:29.374210+0000","last_clean":"2024-04-23T01:52:29.374210+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:52:29.374210+0000","last_undegraded":"2024-04-23T01:52:29.374210+0000","last_fullsized":"2024-04-23T01:52:29.374210+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"48'199","reported_seq":212,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:45.627607+0000","last_change":"2024-04-23T01:52:28.481837+0000","last_active":"2024-04-23T01:52:45.627607+0000","last_peered":"2024-04-23T01:52:45.627607+0000","last_clean":"2024-04-23T01:52:45.627607+0000","last_became_active":"2024-04-23T01:52:28.472032+0000","last_became_peered":"2024-04-23T01:52:28.472032+0000","last_unstale":"2024-04-23T01:52:45.627607+0000","last_undegraded":"2024-04-23T01:52:45.627607+0000","last_fullsized":"2024-04-23T01:52:45.627607+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":200,"num_write_kb":1600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":102,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.628985+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:52:29.628985+0000","last_peered":"2024-04-23T01:52:29.628985+0000","last_clean":"2024-04-23T01:52:29.628985+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:52:29.628985+0000","last_undegraded":"2024-04-23T01:52:29.628985+0000","last_fullsized":"2024-04-23T01:52:29.628985+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"48'199","reported_seq":212,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:44.229164+0000","last_change":"2024-04-23T01:52:28.529295+0000","last_active":"2024-04-23T01:52:44.229164+0000","last_peered":"2024-04-23T01:52:44.229164+0000","last_clean":"2024-04-23T01:52:44.229164+0000","last_became_active":"2024-04-23T01:52:28.523577+0000","last_became_peered":"2024-04-23T01:52:28.523577+0000","last_unstale":"2024-04-23T01:52:44.229164+0000","last_undegraded":"2024-04-23T01:52:44.229164+0000","last_fullsized":"2024-04-23T01:52:44.229164+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":200,"num_write_kb":1600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.368290+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:52:29.368290+0000","last_peered":"2024-04-23T01:52:29.368290+0000","last_clean":"2024-04-23T01:52:29.368290+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:52:29.368290+0000","last_undegraded":"2024-04-23T01:52:29.368290+0000","last_fullsized":"2024-04-23T01:52:29.368290+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"48'151","reported_seq":164,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:46.507216+0000","last_change":"2024-04-23T01:52:28.456738+0000","last_active":"2024-04-23T01:52:46.507216+0000","last_peered":"2024-04-23T01:52:46.507216+0000","last_clean":"2024-04-23T01:52:46.507216+0000","last_became_active":"2024-04-23T01:52:28.432791+0000","last_became_peered":"2024-04-23T01:52:28.432791+0000","last_unstale":"2024-04-23T01:52:46.507216+0000","last_undegraded":"2024-04-23T01:52:46.507216+0000","last_fullsized":"2024-04-23T01:52:46.507216+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":102,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.623888+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:52:29.623888+0000","last_peered":"2024-04-23T01:52:29.623888+0000","last_clean":"2024-04-23T01:52:29.623888+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:52:29.623888+0000","last_undegraded":"2024-04-23T01:52:29.623888+0000","last_fullsized":"2024-04-23T01:52:29.623888+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"48'215","reported_seq":228,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:46.549686+0000","last_change":"2024-04-23T01:52:28.427753+0000","last_active":"2024-04-23T01:52:46.549686+0000","last_peered":"2024-04-23T01:52:46.549686+0000","last_clean":"2024-04-23T01:52:46.549686+0000","last_became_active":"2024-04-23T01:52:28.423538+0000","last_became_peered":"2024-04-23T01:52:28.423538+0000","last_unstale":"2024-04-23T01:52:46.549686+0000","last_undegraded":"2024-04-23T01:52:46.549686+0000","last_fullsized":"2024-04-23T01:52:46.549686+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1769472,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":216,"num_write_kb":1728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":108,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.278611+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:52:29.278611+0000","last_peered":"2024-04-23T01:52:29.278611+0000","last_clean":"2024-04-23T01:52:29.278611+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:52:29.278611+0000","last_undegraded":"2024-04-23T01:52:29.278611+0000","last_fullsized":"2024-04-23T01:52:29.278611+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"48'95","reported_seq":108,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:44.927477+0000","last_change":"2024-04-23T01:52:28.503432+0000","last_active":"2024-04-23T01:52:44.927477+0000","last_peered":"2024-04-23T01:52:44.927477+0000","last_clean":"2024-04-23T01:52:44.927477+0000","last_became_active":"2024-04-23T01:52:28.498953+0000","last_became_peered":"2024-04-23T01:52:28.498953+0000","last_unstale":"2024-04-23T01:52:44.927477+0000","last_undegraded":"2024-04-23T01:52:44.927477+0000","last_fullsized":"2024-04-23T01:52:44.927477+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":786432,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96,"num_write_kb":768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":104,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.389394+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:52:29.389394+0000","last_peered":"2024-04-23T01:52:29.389394+0000","last_clean":"2024-04-23T01:52:29.389394+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:52:29.389394+0000","last_undegraded":"2024-04-23T01:52:29.389394+0000","last_fullsized":"2024-04-23T01:52:29.389394+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"48'115","reported_seq":128,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:46.660062+0000","last_change":"2024-04-23T01:52:28.486992+0000","last_active":"2024-04-23T01:52:46.660062+0000","last_peered":"2024-04-23T01:52:46.660062+0000","last_clean":"2024-04-23T01:52:46.660062+0000","last_became_active":"2024-04-23T01:52:28.482639+0000","last_became_peered":"2024-04-23T01:52:28.482639+0000","last_unstale":"2024-04-23T01:52:46.660062+0000","last_undegraded":"2024-04-23T01:52:46.660062+0000","last_fullsized":"2024-04-23T01:52:46.660062+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":950272,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116,"num_write_kb":928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":108,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.297789+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:52:29.297789+0000","last_peered":"2024-04-23T01:52:29.297789+0000","last_clean":"2024-04-23T01:52:29.297789+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:52:29.297789+0000","last_undegraded":"2024-04-23T01:52:29.297789+0000","last_fullsized":"2024-04-23T01:52:29.297789+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"48'103","reported_seq":116,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:44.859234+0000","last_change":"2024-04-23T01:52:28.453502+0000","last_active":"2024-04-23T01:52:44.859234+0000","last_peered":"2024-04-23T01:52:44.859234+0000","last_clean":"2024-04-23T01:52:44.859234+0000","last_became_active":"2024-04-23T01:52:28.448199+0000","last_became_peered":"2024-04-23T01:52:28.448199+0000","last_unstale":"2024-04-23T01:52:44.859234+0000","last_undegraded":"2024-04-23T01:52:44.859234+0000","last_fullsized":"2024-04-23T01:52:44.859234+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.366780+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:52:29.366780+0000","last_peered":"2024-04-23T01:52:29.366780+0000","last_clean":"2024-04-23T01:52:29.366780+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:52:29.366780+0000","last_undegraded":"2024-04-23T01:52:29.366780+0000","last_fullsized":"2024-04-23T01:52:29.366780+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":20971520,"num_objects":322,"num_object_clones":0,"num_object_copies":966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2560,"num_write_kb":20480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":31436800,"data_stored":62873600,"data_compressed":3243195,"data_compressed_allocated":31436800,"data_compressed_original":62873600,"omap_allocated":0,"internal_metadata":0},"log_size":2544,"ondisk_log_size":2544,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706123,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":99012,"kb_used_data":9724,"kb_used_omap":0,"kb_used_meta":89280,"kb_avail":94272828,"statfs":{"total":96636764160,"available":96535375872,"internally_reserved":0,"allocated":9957376,"data_stored":19700732,"data_compressed":1008128,"data_compressed_allocated":9523200,"data_compressed_original":19537920,"omap_allocated":0,"internal_metadata":91422720},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706123,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":81604,"kb_used_data":6204,"kb_used_omap":0,"kb_used_meta":75392,"kb_avail":94290236,"statfs":{"total":96636764160,"available":96553201664,"internally_reserved":0,"allocated":6352896,"data_stored":12491772,"data_compressed":636330,"data_compressed_allocated":5918720,"data_compressed_original":12328960,"omap_allocated":0,"internal_metadata":77201408},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706123,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":75220,"kb_used_data":8140,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94296620,"statfs":{"total":96636764160,"available":96559738880,"internally_reserved":0,"allocated":8335360,"data_stored":15989212,"data_compressed":817228,"data_compressed_allocated":7921664,"data_compressed_original":15843328,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706123,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":92708,"kb_used_data":7580,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94279132,"statfs":{"total":96636764160,"available":96541831168,"internally_reserved":0,"allocated":7761920,"data_stored":15309820,"data_compressed":781692,"data_compressed_allocated":7327744,"data_compressed_original":15147008,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7442432,"data_stored":14884864,"data_compressed":767813,"data_compressed_allocated":7442432,"data_compressed_original":14884864,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8159232,"data_stored":16318464,"data_compressed":841762,"data_compressed_allocated":8159232,"data_compressed_original":16318464,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6000640,"data_stored":12001280,"data_compressed":619067,"data_compressed_allocated":6000640,"data_compressed_original":12001280,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9834496,"data_stored":19668992,"data_compressed":1014553,"data_compressed_allocated":9834496,"data_compressed_original":19668992,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:52:50.340 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 21 16 3126 3110 1.1569 1.10938 0.0945162 0.107734 2024-04-23T01:52:51.341 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 22 16 3276 3260 1.15757 1.17188 0.0209899 0.107595 2024-04-23T01:52:51.863 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:52:51.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:51.987+0000 7f915ec6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:51.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:51.995+0000 7f915ec6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:52.341 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 23 16 3419 3403 1.15581 1.11719 0.145591 0.107866 2024-04-23T01:52:53.341 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 24 16 3547 3531 1.14932 1 0.0339652 0.108448 2024-04-23T01:52:54.251 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:52:54.341 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 25 16 3704 3688 1.1524 1.22656 0.0756351 0.108143 2024-04-23T01:52:54.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:54.364+0000 7fa6536ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:54.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:54.370+0000 7fa6536ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:55.335 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:52:55.335 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:52:55.336 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:52:55.341 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 26 16 3849 3833 1.15165 1.13281 0.0434159 0.108249 2024-04-23T01:52:55.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:55.449+0000 7f11ef620640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:55.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:55.460+0000 7f11ef620640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:55.760 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:52:55.760 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:52:55.781 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2024-04-23T01:52:55.638894+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27746848,"num_objects":419,"num_object_clones":0,"num_object_copies":1257,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":3452,"num_write_kb":28414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3396,"ondisk_log_size":3396,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":384040,"kb_used_data":40776,"kb_used_omap":0,"kb_used_meta":343232,"kb_avail":377103320,"statfs":{"total":386547056640,"available":386153799680,"internally_reserved":0,"allocated":41754624,"data_stored":82185680,"data_compressed":4208664,"data_compressed_allocated":40038400,"data_compressed_original":81551360,"omap_allocated":0,"internal_metadata":351469568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12271616,"num_objects":189,"num_object_clones":0,"num_object_copies":567,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1498,"num_write_kb":11984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002332"},"pg_stats":[{"pgid":"7.f","version":"48'159","reported_seq":172,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:51.021555+0000","last_change":"2024-04-23T01:52:28.525229+0000","last_active":"2024-04-23T01:52:51.021555+0000","last_peered":"2024-04-23T01:52:51.021555+0000","last_clean":"2024-04-23T01:52:51.021555+0000","last_became_active":"2024-04-23T01:52:28.518971+0000","last_became_peered":"2024-04-23T01:52:28.518971+0000","last_unstale":"2024-04-23T01:52:51.021555+0000","last_undegraded":"2024-04-23T01:52:51.021555+0000","last_fullsized":"2024-04-23T01:52:51.021555+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310720,"num_objects":20,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":160,"num_write_kb":1280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"48'215","reported_seq":228,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:51.288520+0000","last_change":"2024-04-23T01:52:28.530871+0000","last_active":"2024-04-23T01:52:51.288520+0000","last_peered":"2024-04-23T01:52:51.288520+0000","last_clean":"2024-04-23T01:52:51.288520+0000","last_became_active":"2024-04-23T01:52:28.521696+0000","last_became_peered":"2024-04-23T01:52:28.521696+0000","last_unstale":"2024-04-23T01:52:51.288520+0000","last_undegraded":"2024-04-23T01:52:51.288520+0000","last_fullsized":"2024-04-23T01:52:51.288520+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1769472,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":216,"num_write_kb":1728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"48'191","reported_seq":204,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:51.537989+0000","last_change":"2024-04-23T01:52:28.407738+0000","last_active":"2024-04-23T01:52:51.537989+0000","last_peered":"2024-04-23T01:52:51.537989+0000","last_clean":"2024-04-23T01:52:51.537989+0000","last_became_active":"2024-04-23T01:52:28.400739+0000","last_became_peered":"2024-04-23T01:52:28.400739+0000","last_unstale":"2024-04-23T01:52:51.537989+0000","last_undegraded":"2024-04-23T01:52:51.537989+0000","last_fullsized":"2024-04-23T01:52:51.537989+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1572864,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":192,"num_write_kb":1536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"48'175","reported_seq":188,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:51.612008+0000","last_change":"2024-04-23T01:52:28.435954+0000","last_active":"2024-04-23T01:52:51.612008+0000","last_peered":"2024-04-23T01:52:51.612008+0000","last_clean":"2024-04-23T01:52:51.612008+0000","last_became_active":"2024-04-23T01:52:28.430557+0000","last_became_peered":"2024-04-23T01:52:28.430557+0000","last_unstale":"2024-04-23T01:52:51.612008+0000","last_undegraded":"2024-04-23T01:52:51.612008+0000","last_fullsized":"2024-04-23T01:52:51.612008+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"48'199","reported_seq":212,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:50.778657+0000","last_change":"2024-04-23T01:52:28.452034+0000","last_active":"2024-04-23T01:52:50.778657+0000","last_peered":"2024-04-23T01:52:50.778657+0000","last_clean":"2024-04-23T01:52:50.778657+0000","last_became_active":"2024-04-23T01:52:28.430826+0000","last_became_peered":"2024-04-23T01:52:28.430826+0000","last_unstale":"2024-04-23T01:52:50.778657+0000","last_undegraded":"2024-04-23T01:52:50.778657+0000","last_fullsized":"2024-04-23T01:52:50.778657+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":200,"num_write_kb":1600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"48'281","reported_seq":294,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:51.672794+0000","last_change":"2024-04-23T01:52:28.484337+0000","last_active":"2024-04-23T01:52:51.672794+0000","last_peered":"2024-04-23T01:52:51.672794+0000","last_clean":"2024-04-23T01:52:51.672794+0000","last_became_active":"2024-04-23T01:52:28.476566+0000","last_became_peered":"2024-04-23T01:52:28.476566+0000","last_unstale":"2024-04-23T01:52:51.672794+0000","last_undegraded":"2024-04-23T01:52:51.672794+0000","last_fullsized":"2024-04-23T01:52:51.672794+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2310144,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":282,"num_write_kb":2256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"48'225","reported_seq":238,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:51.693972+0000","last_change":"2024-04-23T01:52:28.418531+0000","last_active":"2024-04-23T01:52:51.693972+0000","last_peered":"2024-04-23T01:52:51.693972+0000","last_clean":"2024-04-23T01:52:51.693972+0000","last_became_active":"2024-04-23T01:52:28.412725+0000","last_became_peered":"2024-04-23T01:52:28.412725+0000","last_unstale":"2024-04-23T01:52:51.693972+0000","last_undegraded":"2024-04-23T01:52:51.693972+0000","last_fullsized":"2024-04-23T01:52:51.693972+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851392,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":226,"num_write_kb":1808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"48'231","reported_seq":244,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:51.388937+0000","last_change":"2024-04-23T01:52:28.498385+0000","last_active":"2024-04-23T01:52:51.388937+0000","last_peered":"2024-04-23T01:52:51.388937+0000","last_clean":"2024-04-23T01:52:51.388937+0000","last_became_active":"2024-04-23T01:52:28.487852+0000","last_became_peered":"2024-04-23T01:52:28.487852+0000","last_unstale":"2024-04-23T01:52:51.388937+0000","last_undegraded":"2024-04-23T01:52:51.388937+0000","last_fullsized":"2024-04-23T01:52:51.388937+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1900544,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":232,"num_write_kb":1856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"48'199","reported_seq":212,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:51.481428+0000","last_change":"2024-04-23T01:52:28.536598+0000","last_active":"2024-04-23T01:52:51.481428+0000","last_peered":"2024-04-23T01:52:51.481428+0000","last_clean":"2024-04-23T01:52:51.481428+0000","last_became_active":"2024-04-23T01:52:28.530283+0000","last_became_peered":"2024-04-23T01:52:28.530283+0000","last_unstale":"2024-04-23T01:52:51.481428+0000","last_undegraded":"2024-04-23T01:52:51.481428+0000","last_fullsized":"2024-04-23T01:52:51.481428+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":200,"num_write_kb":1600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":237,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.359805+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:52:29.359805+0000","last_peered":"2024-04-23T01:52:29.359805+0000","last_clean":"2024-04-23T01:52:29.359805+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:52:29.359805+0000","last_undegraded":"2024-04-23T01:52:29.359805+0000","last_fullsized":"2024-04-23T01:52:29.359805+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.374210+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:52:29.374210+0000","last_peered":"2024-04-23T01:52:29.374210+0000","last_clean":"2024-04-23T01:52:29.374210+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:52:29.374210+0000","last_undegraded":"2024-04-23T01:52:29.374210+0000","last_fullsized":"2024-04-23T01:52:29.374210+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"48'215","reported_seq":228,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:51.480884+0000","last_change":"2024-04-23T01:52:28.481837+0000","last_active":"2024-04-23T01:52:51.480884+0000","last_peered":"2024-04-23T01:52:51.480884+0000","last_clean":"2024-04-23T01:52:51.480884+0000","last_became_active":"2024-04-23T01:52:28.472032+0000","last_became_peered":"2024-04-23T01:52:28.472032+0000","last_unstale":"2024-04-23T01:52:51.480884+0000","last_undegraded":"2024-04-23T01:52:51.480884+0000","last_fullsized":"2024-04-23T01:52:51.480884+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1769472,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":216,"num_write_kb":1728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":102,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.628985+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:52:29.628985+0000","last_peered":"2024-04-23T01:52:29.628985+0000","last_clean":"2024-04-23T01:52:29.628985+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:52:29.628985+0000","last_undegraded":"2024-04-23T01:52:29.628985+0000","last_fullsized":"2024-04-23T01:52:29.628985+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"48'247","reported_seq":260,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:50.842909+0000","last_change":"2024-04-23T01:52:28.529295+0000","last_active":"2024-04-23T01:52:50.842909+0000","last_peered":"2024-04-23T01:52:50.842909+0000","last_clean":"2024-04-23T01:52:50.842909+0000","last_became_active":"2024-04-23T01:52:28.523577+0000","last_became_peered":"2024-04-23T01:52:28.523577+0000","last_unstale":"2024-04-23T01:52:50.842909+0000","last_undegraded":"2024-04-23T01:52:50.842909+0000","last_fullsized":"2024-04-23T01:52:50.842909+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.368290+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:52:29.368290+0000","last_peered":"2024-04-23T01:52:29.368290+0000","last_clean":"2024-04-23T01:52:29.368290+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:52:29.368290+0000","last_undegraded":"2024-04-23T01:52:29.368290+0000","last_fullsized":"2024-04-23T01:52:29.368290+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"48'212","reported_seq":225,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:51.608497+0000","last_change":"2024-04-23T01:52:28.456738+0000","last_active":"2024-04-23T01:52:51.608497+0000","last_peered":"2024-04-23T01:52:51.608497+0000","last_clean":"2024-04-23T01:52:51.608497+0000","last_became_active":"2024-04-23T01:52:28.432791+0000","last_became_peered":"2024-04-23T01:52:28.432791+0000","last_unstale":"2024-04-23T01:52:51.608497+0000","last_undegraded":"2024-04-23T01:52:51.608497+0000","last_fullsized":"2024-04-23T01:52:51.608497+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1744896,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":213,"num_write_kb":1704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":102,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.623888+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:52:29.623888+0000","last_peered":"2024-04-23T01:52:29.623888+0000","last_clean":"2024-04-23T01:52:29.623888+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:52:29.623888+0000","last_undegraded":"2024-04-23T01:52:29.623888+0000","last_fullsized":"2024-04-23T01:52:29.623888+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"48'305","reported_seq":318,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:51.593332+0000","last_change":"2024-04-23T01:52:28.427753+0000","last_active":"2024-04-23T01:52:51.593332+0000","last_peered":"2024-04-23T01:52:51.593332+0000","last_clean":"2024-04-23T01:52:51.593332+0000","last_became_active":"2024-04-23T01:52:28.423538+0000","last_became_peered":"2024-04-23T01:52:28.423538+0000","last_unstale":"2024-04-23T01:52:51.593332+0000","last_undegraded":"2024-04-23T01:52:51.593332+0000","last_fullsized":"2024-04-23T01:52:51.593332+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2506752,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":306,"num_write_kb":2448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":108,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.278611+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:52:29.278611+0000","last_peered":"2024-04-23T01:52:29.278611+0000","last_clean":"2024-04-23T01:52:29.278611+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:52:29.278611+0000","last_undegraded":"2024-04-23T01:52:29.278611+0000","last_fullsized":"2024-04-23T01:52:29.278611+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"48'151","reported_seq":164,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:51.058298+0000","last_change":"2024-04-23T01:52:28.503432+0000","last_active":"2024-04-23T01:52:51.058298+0000","last_peered":"2024-04-23T01:52:51.058298+0000","last_clean":"2024-04-23T01:52:51.058298+0000","last_became_active":"2024-04-23T01:52:28.498953+0000","last_became_peered":"2024-04-23T01:52:28.498953+0000","last_unstale":"2024-04-23T01:52:51.058298+0000","last_undegraded":"2024-04-23T01:52:51.058298+0000","last_fullsized":"2024-04-23T01:52:51.058298+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":104,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.389394+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:52:29.389394+0000","last_peered":"2024-04-23T01:52:29.389394+0000","last_clean":"2024-04-23T01:52:29.389394+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:52:29.389394+0000","last_undegraded":"2024-04-23T01:52:29.389394+0000","last_fullsized":"2024-04-23T01:52:29.389394+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"48'151","reported_seq":164,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:51.181162+0000","last_change":"2024-04-23T01:52:28.486992+0000","last_active":"2024-04-23T01:52:51.181162+0000","last_peered":"2024-04-23T01:52:51.181162+0000","last_clean":"2024-04-23T01:52:51.181162+0000","last_became_active":"2024-04-23T01:52:28.482639+0000","last_became_peered":"2024-04-23T01:52:28.482639+0000","last_unstale":"2024-04-23T01:52:51.181162+0000","last_undegraded":"2024-04-23T01:52:51.181162+0000","last_fullsized":"2024-04-23T01:52:51.181162+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":108,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.297789+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:52:29.297789+0000","last_peered":"2024-04-23T01:52:29.297789+0000","last_clean":"2024-04-23T01:52:29.297789+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:52:29.297789+0000","last_undegraded":"2024-04-23T01:52:29.297789+0000","last_fullsized":"2024-04-23T01:52:29.297789+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"48'143","reported_seq":156,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:51.038812+0000","last_change":"2024-04-23T01:52:28.453502+0000","last_active":"2024-04-23T01:52:51.038812+0000","last_peered":"2024-04-23T01:52:51.038812+0000","last_clean":"2024-04-23T01:52:51.038812+0000","last_became_active":"2024-04-23T01:52:28.448199+0000","last_became_peered":"2024-04-23T01:52:28.448199+0000","last_unstale":"2024-04-23T01:52:51.038812+0000","last_undegraded":"2024-04-23T01:52:51.038812+0000","last_fullsized":"2024-04-23T01:52:51.038812+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.366780+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:52:29.366780+0000","last_peered":"2024-04-23T01:52:29.366780+0000","last_clean":"2024-04-23T01:52:29.366780+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:52:29.366780+0000","last_undegraded":"2024-04-23T01:52:29.366780+0000","last_fullsized":"2024-04-23T01:52:29.366780+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":27156480,"num_objects":417,"num_object_clones":0,"num_object_copies":1251,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3315,"num_write_kb":26520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40722432,"data_stored":81444864,"data_compressed":4202136,"data_compressed_allocated":40722432,"data_compressed_original":81444864,"omap_allocated":0,"internal_metadata":0},"log_size":3299,"ondisk_log_size":3299,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706128,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":106032,"kb_used_data":12584,"kb_used_omap":0,"kb_used_meta":93440,"kb_avail":94265808,"statfs":{"total":96636764160,"available":96528187392,"internally_reserved":0,"allocated":12886016,"data_stored":25558012,"data_compressed":1310573,"data_compressed_allocated":12451840,"data_compressed_original":25395200,"omap_allocated":0,"internal_metadata":95682560},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706128,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":83120,"kb_used_data":7720,"kb_used_omap":0,"kb_used_meta":75392,"kb_avail":94288720,"statfs":{"total":96636764160,"available":96551649280,"internally_reserved":0,"allocated":7905280,"data_stored":15596540,"data_compressed":796647,"data_compressed_allocated":7471104,"data_compressed_original":15433728,"omap_allocated":0,"internal_metadata":77201408},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706128,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":95840,"kb_used_data":10712,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94276000,"statfs":{"total":96636764160,"available":96538624000,"internally_reserved":0,"allocated":10969088,"data_stored":21256668,"data_compressed":1089217,"data_compressed_allocated":10555392,"data_compressed_original":21110784,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706128,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":99048,"kb_used_data":9760,"kb_used_omap":0,"kb_used_meta":89280,"kb_avail":94272792,"statfs":{"total":96636764160,"available":96535339008,"internally_reserved":0,"allocated":9994240,"data_stored":19774460,"data_compressed":1012227,"data_compressed_allocated":9560064,"data_compressed_original":19611648,"omap_allocated":0,"internal_metadata":91422720},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9621504,"data_stored":19243008,"data_compressed":992849,"data_compressed_allocated":9621504,"data_compressed_original":19243008,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10776576,"data_stored":21553152,"data_compressed":1112059,"data_compressed_allocated":10776576,"data_compressed_original":21553152,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7618560,"data_stored":15237120,"data_compressed":786152,"data_compressed_allocated":7618560,"data_compressed_original":15237120,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12705792,"data_stored":25411584,"data_compressed":1311076,"data_compressed_allocated":12705792,"data_compressed_original":25411584,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:52:56.341 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 27 16 3998 3982 1.1521 1.16406 0.181632 0.108197 2024-04-23T01:52:56.609 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:52:56.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:56.757+0000 7fd81dc64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:56.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:56.763+0000 7fd81dc64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:57.341 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 28 16 4152 4136 1.15392 1.20312 0.209314 0.108088 2024-04-23T01:52:58.341 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 29 16 4301 4285 1.15427 1.16406 0.164404 0.108026 2024-04-23T01:52:59.069 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:52:59.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:59.209+0000 7f7c75824640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:59.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:52:59.215+0000 7f7c75824640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:52:59.341 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 30 16 4444 4428 1.15303 1.11719 0.0539981 0.108111 2024-04-23T01:53:00.342 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 31 16 4596 4580 1.15414 1.1875 0.213837 0.10805 2024-04-23T01:53:00.783 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:53:00.783 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:53:00.783 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:53:00.784 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:53:00.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:00.895+0000 7f9b0deed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:00.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:00.905+0000 7f9b0deed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:00.922 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:53:00.922 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:53:00.922 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:53:00.922 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:53:00.922 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:53:00.922 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:53:00.923 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:53:00.923 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:53:00.923 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:53:00.923 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:53:00.923 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:53:00.923 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:53:00.925 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:53:00.926 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:53:00.926 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:53:01.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:01.038+0000 7fc525b54640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:01.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:01.051+0000 7fc525b54640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:01.097 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:53:01.097 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:53:01.097 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:53:01.097 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:53:01.097 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:53:01.097 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:53:01.097 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:53:01.097 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:53:01.097 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:53:01.098 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:53:01.098 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:53:01.098 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:53:01.100 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:53:01.101 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:53:01.101 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:53:01.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:01.218+0000 7fbd27e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:01.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:01.231+0000 7fbd27e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:01.252 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:53:01.253 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:53:01.253 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:53:01.253 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:53:01.253 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:53:01.253 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:53:01.253 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:53:01.253 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:53:01.253 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:53:01.253 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:53:01.253 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:53:01.253 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:53:01.256 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:53:01.257 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:53:01.257 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:53:01.342 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 32 16 4740 4724 1.15322 1.125 0.0653678 0.108121 2024-04-23T01:53:01.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:01.382+0000 7f60f74ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:01.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:01.392+0000 7f60f74ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:01.442 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:53:01.509 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:53:01.510 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:53:01.510 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:53:01.510 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:53:01.510 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:53:01.510 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:53:01.510 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:53:01.510 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:53:01.510 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:53:01.510 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:53:01.510 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:53:01.510 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:53:01.513 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:53:01.513 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:53:01.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:01.563+0000 7fef3f4ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:01.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:01.571+0000 7fef3f4ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:02.342 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 33 16 4887 4871 1.15307 1.14844 0.095051 0.108183 2024-04-23T01:53:03.342 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 34 16 5043 5027 1.155 1.21875 0.0510573 0.108053 2024-04-23T01:53:03.781 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:53:03.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:03.900+0000 7f7f5bd9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:03.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:03.907+0000 7f7f5bd9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:04.342 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 35 16 5188 5172 1.15436 1.13281 0.174572 0.108069 2024-04-23T01:53:05.342 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 36 16 5332 5316 1.15355 1.125 0.131648 0.108082 2024-04-23T01:53:06.106 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:53:06.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:06.231+0000 7f7b0ac2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:06.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:06.238+0000 7f7b0ac2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:06.342 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 37 16 5465 5449 1.15045 1.03906 0.181408 0.108333 2024-04-23T01:53:06.514 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:53:06.514 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:53:06.514 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:53:06.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:06.627+0000 7f57c9f5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:06.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:06.635+0000 7f57c9f5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:06.839 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:53:06.839 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:53:06.853 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":281,"stamp":"2024-04-23T01:53:05.640742+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":39715360,"num_objects":601,"num_object_clones":0,"num_object_copies":1803,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":4913,"num_write_kb":40102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4857,"ondisk_log_size":4857,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":393284,"kb_used_data":58340,"kb_used_omap":0,"kb_used_meta":334912,"kb_avail":377094076,"statfs":{"total":386547056640,"available":386144333824,"internally_reserved":0,"allocated":59740160,"data_stored":118156752,"data_compressed":6066057,"data_compressed_allocated":58023936,"data_compressed_original":117522432,"omap_allocated":0,"internal_metadata":342949888},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11968512,"num_objects":182,"num_object_clones":0,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1461,"num_write_kb":11688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002349"},"pg_stats":[{"pgid":"7.f","version":"48'287","reported_seq":300,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:00.566290+0000","last_change":"2024-04-23T01:52:28.525229+0000","last_active":"2024-04-23T01:53:00.566290+0000","last_peered":"2024-04-23T01:53:00.566290+0000","last_clean":"2024-04-23T01:53:00.566290+0000","last_became_active":"2024-04-23T01:52:28.518971+0000","last_became_peered":"2024-04-23T01:52:28.518971+0000","last_unstale":"2024-04-23T01:53:00.566290+0000","last_undegraded":"2024-04-23T01:53:00.566290+0000","last_fullsized":"2024-04-23T01:53:00.566290+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"48'295","reported_seq":308,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:57.920361+0000","last_change":"2024-04-23T01:52:28.530871+0000","last_active":"2024-04-23T01:52:57.920361+0000","last_peered":"2024-04-23T01:52:57.920361+0000","last_clean":"2024-04-23T01:52:57.920361+0000","last_became_active":"2024-04-23T01:52:28.521696+0000","last_became_peered":"2024-04-23T01:52:28.521696+0000","last_unstale":"2024-04-23T01:52:57.920361+0000","last_undegraded":"2024-04-23T01:52:57.920361+0000","last_fullsized":"2024-04-23T01:52:57.920361+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2424832,"num_objects":37,"num_object_clones":0,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":296,"num_write_kb":2368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"48'287","reported_seq":300,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:59.902472+0000","last_change":"2024-04-23T01:52:28.407738+0000","last_active":"2024-04-23T01:52:59.902472+0000","last_peered":"2024-04-23T01:52:59.902472+0000","last_clean":"2024-04-23T01:52:59.902472+0000","last_became_active":"2024-04-23T01:52:28.400739+0000","last_became_peered":"2024-04-23T01:52:28.400739+0000","last_unstale":"2024-04-23T01:52:59.902472+0000","last_undegraded":"2024-04-23T01:52:59.902472+0000","last_fullsized":"2024-04-23T01:52:59.902472+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"48'239","reported_seq":252,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:00.890410+0000","last_change":"2024-04-23T01:52:28.435954+0000","last_active":"2024-04-23T01:53:00.890410+0000","last_peered":"2024-04-23T01:53:00.890410+0000","last_clean":"2024-04-23T01:53:00.890410+0000","last_became_active":"2024-04-23T01:52:28.430557+0000","last_became_peered":"2024-04-23T01:52:28.430557+0000","last_unstale":"2024-04-23T01:53:00.890410+0000","last_undegraded":"2024-04-23T01:53:00.890410+0000","last_fullsized":"2024-04-23T01:53:00.890410+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":240,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"48'263","reported_seq":276,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:01.645671+0000","last_change":"2024-04-23T01:52:28.452034+0000","last_active":"2024-04-23T01:53:01.645671+0000","last_peered":"2024-04-23T01:53:01.645671+0000","last_clean":"2024-04-23T01:53:01.645671+0000","last_became_active":"2024-04-23T01:52:28.430826+0000","last_became_peered":"2024-04-23T01:52:28.430826+0000","last_unstale":"2024-04-23T01:53:01.645671+0000","last_undegraded":"2024-04-23T01:53:01.645671+0000","last_fullsized":"2024-04-23T01:53:01.645671+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162688,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":264,"num_write_kb":2112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"48'391","reported_seq":404,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:01.401631+0000","last_change":"2024-04-23T01:52:28.484337+0000","last_active":"2024-04-23T01:53:01.401631+0000","last_peered":"2024-04-23T01:53:01.401631+0000","last_clean":"2024-04-23T01:53:01.401631+0000","last_became_active":"2024-04-23T01:52:28.476566+0000","last_became_peered":"2024-04-23T01:52:28.476566+0000","last_unstale":"2024-04-23T01:53:01.401631+0000","last_undegraded":"2024-04-23T01:53:01.401631+0000","last_fullsized":"2024-04-23T01:53:01.401631+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3211264,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":392,"num_write_kb":3136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"48'287","reported_seq":300,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:01.223336+0000","last_change":"2024-04-23T01:52:28.418531+0000","last_active":"2024-04-23T01:53:01.223336+0000","last_peered":"2024-04-23T01:53:01.223336+0000","last_clean":"2024-04-23T01:53:01.223336+0000","last_became_active":"2024-04-23T01:52:28.412725+0000","last_became_peered":"2024-04-23T01:52:28.412725+0000","last_unstale":"2024-04-23T01:53:01.223336+0000","last_undegraded":"2024-04-23T01:53:01.223336+0000","last_fullsized":"2024-04-23T01:53:01.223336+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"48'351","reported_seq":364,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:01.059159+0000","last_change":"2024-04-23T01:52:28.498385+0000","last_active":"2024-04-23T01:53:01.059159+0000","last_peered":"2024-04-23T01:53:01.059159+0000","last_clean":"2024-04-23T01:53:01.059159+0000","last_became_active":"2024-04-23T01:52:28.487852+0000","last_became_peered":"2024-04-23T01:52:28.487852+0000","last_unstale":"2024-04-23T01:53:01.059159+0000","last_undegraded":"2024-04-23T01:53:01.059159+0000","last_fullsized":"2024-04-23T01:53:01.059159+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883584,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":352,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"48'255","reported_seq":268,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:59.924537+0000","last_change":"2024-04-23T01:52:28.536598+0000","last_active":"2024-04-23T01:52:59.924537+0000","last_peered":"2024-04-23T01:52:59.924537+0000","last_clean":"2024-04-23T01:52:59.924537+0000","last_became_active":"2024-04-23T01:52:28.530283+0000","last_became_peered":"2024-04-23T01:52:28.530283+0000","last_unstale":"2024-04-23T01:52:59.924537+0000","last_undegraded":"2024-04-23T01:52:59.924537+0000","last_fullsized":"2024-04-23T01:52:59.924537+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":32,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":256,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":237,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.359805+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:52:29.359805+0000","last_peered":"2024-04-23T01:52:29.359805+0000","last_clean":"2024-04-23T01:52:29.359805+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:52:29.359805+0000","last_undegraded":"2024-04-23T01:52:29.359805+0000","last_fullsized":"2024-04-23T01:52:29.359805+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.374210+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:52:29.374210+0000","last_peered":"2024-04-23T01:52:29.374210+0000","last_clean":"2024-04-23T01:52:29.374210+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:52:29.374210+0000","last_undegraded":"2024-04-23T01:52:29.374210+0000","last_fullsized":"2024-04-23T01:52:29.374210+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"48'343","reported_seq":356,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:01.580579+0000","last_change":"2024-04-23T01:52:28.481837+0000","last_active":"2024-04-23T01:53:01.580579+0000","last_peered":"2024-04-23T01:53:01.580579+0000","last_clean":"2024-04-23T01:53:01.580579+0000","last_became_active":"2024-04-23T01:52:28.472032+0000","last_became_peered":"2024-04-23T01:52:28.472032+0000","last_unstale":"2024-04-23T01:53:01.580579+0000","last_undegraded":"2024-04-23T01:53:01.580579+0000","last_fullsized":"2024-04-23T01:53:01.580579+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":344,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":102,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.628985+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:52:29.628985+0000","last_peered":"2024-04-23T01:52:29.628985+0000","last_clean":"2024-04-23T01:52:29.628985+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:52:29.628985+0000","last_undegraded":"2024-04-23T01:52:29.628985+0000","last_fullsized":"2024-04-23T01:52:29.628985+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"48'351","reported_seq":364,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:01.594706+0000","last_change":"2024-04-23T01:52:28.529295+0000","last_active":"2024-04-23T01:53:01.594706+0000","last_peered":"2024-04-23T01:53:01.594706+0000","last_clean":"2024-04-23T01:53:01.594706+0000","last_became_active":"2024-04-23T01:52:28.523577+0000","last_became_peered":"2024-04-23T01:52:28.523577+0000","last_unstale":"2024-04-23T01:53:01.594706+0000","last_undegraded":"2024-04-23T01:53:01.594706+0000","last_fullsized":"2024-04-23T01:53:01.594706+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883584,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":352,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.368290+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:52:29.368290+0000","last_peered":"2024-04-23T01:52:29.368290+0000","last_clean":"2024-04-23T01:52:29.368290+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:52:29.368290+0000","last_undegraded":"2024-04-23T01:52:29.368290+0000","last_fullsized":"2024-04-23T01:52:29.368290+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"48'257","reported_seq":270,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:01.604877+0000","last_change":"2024-04-23T01:52:28.456738+0000","last_active":"2024-04-23T01:53:01.604877+0000","last_peered":"2024-04-23T01:53:01.604877+0000","last_clean":"2024-04-23T01:53:01.604877+0000","last_became_active":"2024-04-23T01:52:28.432791+0000","last_became_peered":"2024-04-23T01:52:28.432791+0000","last_unstale":"2024-04-23T01:53:01.604877+0000","last_undegraded":"2024-04-23T01:53:01.604877+0000","last_fullsized":"2024-04-23T01:53:01.604877+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2113536,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":258,"num_write_kb":2064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":102,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.623888+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:52:29.623888+0000","last_peered":"2024-04-23T01:52:29.623888+0000","last_clean":"2024-04-23T01:52:29.623888+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:52:29.623888+0000","last_undegraded":"2024-04-23T01:52:29.623888+0000","last_fullsized":"2024-04-23T01:52:29.623888+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"48'427","reported_seq":440,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:01.597173+0000","last_change":"2024-04-23T01:52:28.427753+0000","last_active":"2024-04-23T01:53:01.597173+0000","last_peered":"2024-04-23T01:53:01.597173+0000","last_clean":"2024-04-23T01:53:01.597173+0000","last_became_active":"2024-04-23T01:52:28.423538+0000","last_became_peered":"2024-04-23T01:52:28.423538+0000","last_unstale":"2024-04-23T01:53:01.597173+0000","last_undegraded":"2024-04-23T01:53:01.597173+0000","last_fullsized":"2024-04-23T01:53:01.597173+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3506176,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":428,"num_write_kb":3424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":108,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.278611+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:52:29.278611+0000","last_peered":"2024-04-23T01:52:29.278611+0000","last_clean":"2024-04-23T01:52:29.278611+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:52:29.278611+0000","last_undegraded":"2024-04-23T01:52:29.278611+0000","last_fullsized":"2024-04-23T01:52:29.278611+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"48'239","reported_seq":252,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:01.473796+0000","last_change":"2024-04-23T01:52:28.503432+0000","last_active":"2024-04-23T01:53:01.473796+0000","last_peered":"2024-04-23T01:53:01.473796+0000","last_clean":"2024-04-23T01:53:01.473796+0000","last_became_active":"2024-04-23T01:52:28.498953+0000","last_became_peered":"2024-04-23T01:52:28.498953+0000","last_unstale":"2024-04-23T01:53:01.473796+0000","last_undegraded":"2024-04-23T01:53:01.473796+0000","last_fullsized":"2024-04-23T01:53:01.473796+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":240,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":104,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.389394+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:52:29.389394+0000","last_peered":"2024-04-23T01:52:29.389394+0000","last_clean":"2024-04-23T01:52:29.389394+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:52:29.389394+0000","last_undegraded":"2024-04-23T01:52:29.389394+0000","last_fullsized":"2024-04-23T01:52:29.389394+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"48'247","reported_seq":260,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:01.120023+0000","last_change":"2024-04-23T01:52:28.486992+0000","last_active":"2024-04-23T01:53:01.120023+0000","last_peered":"2024-04-23T01:53:01.120023+0000","last_clean":"2024-04-23T01:53:01.120023+0000","last_became_active":"2024-04-23T01:52:28.482639+0000","last_became_peered":"2024-04-23T01:52:28.482639+0000","last_unstale":"2024-04-23T01:53:01.120023+0000","last_undegraded":"2024-04-23T01:53:01.120023+0000","last_fullsized":"2024-04-23T01:53:01.120023+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":108,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.297789+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:52:29.297789+0000","last_peered":"2024-04-23T01:52:29.297789+0000","last_clean":"2024-04-23T01:52:29.297789+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:52:29.297789+0000","last_undegraded":"2024-04-23T01:52:29.297789+0000","last_fullsized":"2024-04-23T01:52:29.297789+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"48'241","reported_seq":254,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:01.648826+0000","last_change":"2024-04-23T01:52:28.453502+0000","last_active":"2024-04-23T01:53:01.648826+0000","last_peered":"2024-04-23T01:53:01.648826+0000","last_clean":"2024-04-23T01:53:01.648826+0000","last_became_active":"2024-04-23T01:52:28.448199+0000","last_became_peered":"2024-04-23T01:52:28.448199+0000","last_unstale":"2024-04-23T01:53:01.648826+0000","last_undegraded":"2024-04-23T01:53:01.648826+0000","last_fullsized":"2024-04-23T01:53:01.648826+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1982464,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":242,"num_write_kb":1936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.366780+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:52:29.366780+0000","last_peered":"2024-04-23T01:52:29.366780+0000","last_clean":"2024-04-23T01:52:29.366780+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:52:29.366780+0000","last_undegraded":"2024-04-23T01:52:29.366780+0000","last_fullsized":"2024-04-23T01:52:29.366780+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":39124992,"num_objects":599,"num_object_clones":0,"num_object_copies":1797,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4776,"num_write_kb":38208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":58662912,"data_stored":117325824,"data_compressed":6054876,"data_compressed_allocated":58662912,"data_compressed_original":117325824,"omap_allocated":0,"internal_metadata":0},"log_size":4760,"ondisk_log_size":4760,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706138,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":103224,"kb_used_data":18096,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94268616,"statfs":{"total":96636764160,"available":96531062784,"internally_reserved":0,"allocated":18530304,"data_stored":36846588,"data_compressed":1893467,"data_compressed_allocated":18096128,"data_compressed_original":36683776,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706138,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":73964,"kb_used_data":11044,"kb_used_omap":0,"kb_used_meta":62912,"kb_avail":94297876,"statfs":{"total":96636764160,"available":96561025024,"internally_reserved":0,"allocated":11309056,"data_stored":22404092,"data_compressed":1148160,"data_compressed_allocated":10874880,"data_compressed_original":22241280,"omap_allocated":0,"internal_metadata":64421888},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706138,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":108800,"kb_used_data":15352,"kb_used_omap":0,"kb_used_meta":93440,"kb_avail":94263040,"statfs":{"total":96636764160,"available":96525352960,"internally_reserved":0,"allocated":15720448,"data_stored":30759388,"data_compressed":1579897,"data_compressed_allocated":15306752,"data_compressed_original":30613504,"omap_allocated":0,"internal_metadata":95682560},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706138,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":107296,"kb_used_data":13848,"kb_used_omap":0,"kb_used_meta":93440,"kb_avail":94264544,"statfs":{"total":96636764160,"available":96526893056,"internally_reserved":0,"allocated":14180352,"data_stored":28146684,"data_compressed":1444533,"data_compressed_allocated":13746176,"data_compressed_original":27983872,"omap_allocated":0,"internal_metadata":95682560},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13811712,"data_stored":27623424,"data_compressed":1425578,"data_compressed_allocated":13811712,"data_compressed_original":27623424,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15478784,"data_stored":30957568,"data_compressed":1597663,"data_compressed_allocated":15478784,"data_compressed_original":30957568,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":11010048,"data_stored":22020096,"data_compressed":1136396,"data_compressed_allocated":11010048,"data_compressed_original":22020096,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":18362368,"data_stored":36724736,"data_compressed":1895239,"data_compressed_allocated":18362368,"data_compressed_original":36724736,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:53:07.342 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 38 16 5618 5602 1.15163 1.19531 0.0476698 0.108378 2024-04-23T01:53:08.343 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 39 16 5756 5740 1.14974 1.07812 0.155721 0.108501 2024-04-23T01:53:08.428 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:53:08.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:08.542+0000 7f66a835d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:08.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:08.549+0000 7f66a835d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:09.346 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:53:09.342799+0000 min lat: 0.0154323 max lat: 0.370563 avg lat: 0.10875 2024-04-23T01:53:09.346 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:53:09.346 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 40 16 5892 5876 1.14756 1.0625 0.0586102 0.10875 2024-04-23T01:53:10.343 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 41 16 6021 6005 1.14415 1.00781 0.146763 0.10904 2024-04-23T01:53:10.733 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:53:10.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:10.844+0000 7f6311bff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:10.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:10.853+0000 7f631099d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:11.343 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 42 16 6158 6142 1.14239 1.07031 0.0786967 0.109263 2024-04-23T01:53:11.855 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:53:11.855 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:53:11.856 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:53:11.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:11.976+0000 7fa29642d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:11.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:11.985+0000 7fa29642d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:12.203 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:53:12.203 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:53:12.219 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2024-04-23T01:53:11.641914+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":45654560,"num_objects":691,"num_object_clones":0,"num_object_copies":2073,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":5638,"num_write_kb":45902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5582,"ondisk_log_size":5582,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":424256,"kb_used_data":67104,"kb_used_omap":0,"kb_used_meta":357120,"kb_avail":377063104,"statfs":{"total":386547056640,"available":386112618496,"internally_reserved":0,"allocated":68714496,"data_stored":136105424,"data_compressed":6992850,"data_compressed_allocated":66998272,"data_compressed_original":135471104,"omap_allocated":0,"internal_metadata":365690880},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11984896,"num_objects":182,"num_object_clones":0,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1463,"num_write_kb":11704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002245"},"pg_stats":[{"pgid":"7.f","version":"48'343","reported_seq":356,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:06.645233+0000","last_change":"2024-04-23T01:52:28.525229+0000","last_active":"2024-04-23T01:53:06.645233+0000","last_peered":"2024-04-23T01:53:06.645233+0000","last_clean":"2024-04-23T01:53:06.645233+0000","last_became_active":"2024-04-23T01:52:28.518971+0000","last_became_peered":"2024-04-23T01:52:28.518971+0000","last_unstale":"2024-04-23T01:53:06.645233+0000","last_undegraded":"2024-04-23T01:53:06.645233+0000","last_fullsized":"2024-04-23T01:53:06.645233+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":344,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"48'356","reported_seq":369,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:06.623856+0000","last_change":"2024-04-23T01:52:28.530871+0000","last_active":"2024-04-23T01:53:06.623856+0000","last_peered":"2024-04-23T01:53:06.623856+0000","last_clean":"2024-04-23T01:53:06.623856+0000","last_became_active":"2024-04-23T01:52:28.521696+0000","last_became_peered":"2024-04-23T01:52:28.521696+0000","last_unstale":"2024-04-23T01:53:06.623856+0000","last_undegraded":"2024-04-23T01:53:06.623856+0000","last_fullsized":"2024-04-23T01:53:06.623856+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2924544,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":357,"num_write_kb":2856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"48'327","reported_seq":340,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:04.885343+0000","last_change":"2024-04-23T01:52:28.407738+0000","last_active":"2024-04-23T01:53:04.885343+0000","last_peered":"2024-04-23T01:53:04.885343+0000","last_clean":"2024-04-23T01:53:04.885343+0000","last_became_active":"2024-04-23T01:52:28.400739+0000","last_became_peered":"2024-04-23T01:52:28.400739+0000","last_unstale":"2024-04-23T01:53:04.885343+0000","last_undegraded":"2024-04-23T01:53:04.885343+0000","last_fullsized":"2024-04-23T01:53:04.885343+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2686976,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":328,"num_write_kb":2624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"48'287","reported_seq":300,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:05.918902+0000","last_change":"2024-04-23T01:52:28.435954+0000","last_active":"2024-04-23T01:53:05.918902+0000","last_peered":"2024-04-23T01:53:05.918902+0000","last_clean":"2024-04-23T01:53:05.918902+0000","last_became_active":"2024-04-23T01:52:28.430557+0000","last_became_peered":"2024-04-23T01:52:28.430557+0000","last_unstale":"2024-04-23T01:53:05.918902+0000","last_undegraded":"2024-04-23T01:53:05.918902+0000","last_fullsized":"2024-04-23T01:53:05.918902+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"48'287","reported_seq":300,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:04.917132+0000","last_change":"2024-04-23T01:52:28.452034+0000","last_active":"2024-04-23T01:53:04.917132+0000","last_peered":"2024-04-23T01:53:04.917132+0000","last_clean":"2024-04-23T01:53:04.917132+0000","last_became_active":"2024-04-23T01:52:28.430826+0000","last_became_peered":"2024-04-23T01:52:28.430826+0000","last_unstale":"2024-04-23T01:53:04.917132+0000","last_undegraded":"2024-04-23T01:53:04.917132+0000","last_fullsized":"2024-04-23T01:53:04.917132+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"48'423","reported_seq":436,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:05.719925+0000","last_change":"2024-04-23T01:52:28.484337+0000","last_active":"2024-04-23T01:53:05.719925+0000","last_peered":"2024-04-23T01:53:05.719925+0000","last_clean":"2024-04-23T01:53:05.719925+0000","last_became_active":"2024-04-23T01:52:28.476566+0000","last_became_peered":"2024-04-23T01:52:28.476566+0000","last_unstale":"2024-04-23T01:53:05.719925+0000","last_undegraded":"2024-04-23T01:53:05.719925+0000","last_fullsized":"2024-04-23T01:53:05.719925+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3473408,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":424,"num_write_kb":3392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"48'316","reported_seq":329,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:06.695852+0000","last_change":"2024-04-23T01:52:28.418531+0000","last_active":"2024-04-23T01:53:06.695852+0000","last_peered":"2024-04-23T01:53:06.695852+0000","last_clean":"2024-04-23T01:53:06.695852+0000","last_became_active":"2024-04-23T01:52:28.412725+0000","last_became_peered":"2024-04-23T01:52:28.412725+0000","last_unstale":"2024-04-23T01:53:06.695852+0000","last_undegraded":"2024-04-23T01:53:06.695852+0000","last_fullsized":"2024-04-23T01:53:06.695852+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2596864,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":317,"num_write_kb":2536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"48'399","reported_seq":412,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:06.238686+0000","last_change":"2024-04-23T01:52:28.498385+0000","last_active":"2024-04-23T01:53:06.238686+0000","last_peered":"2024-04-23T01:53:06.238686+0000","last_clean":"2024-04-23T01:53:06.238686+0000","last_became_active":"2024-04-23T01:52:28.487852+0000","last_became_peered":"2024-04-23T01:52:28.487852+0000","last_unstale":"2024-04-23T01:53:06.238686+0000","last_undegraded":"2024-04-23T01:53:06.238686+0000","last_fullsized":"2024-04-23T01:53:06.238686+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3276800,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":400,"num_write_kb":3200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"48'319","reported_seq":332,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:06.534386+0000","last_change":"2024-04-23T01:52:28.536598+0000","last_active":"2024-04-23T01:53:06.534386+0000","last_peered":"2024-04-23T01:53:06.534386+0000","last_clean":"2024-04-23T01:53:06.534386+0000","last_became_active":"2024-04-23T01:52:28.530283+0000","last_became_peered":"2024-04-23T01:52:28.530283+0000","last_unstale":"2024-04-23T01:53:06.534386+0000","last_undegraded":"2024-04-23T01:53:06.534386+0000","last_fullsized":"2024-04-23T01:53:06.534386+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":320,"num_write_kb":2560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":237,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.359805+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:52:29.359805+0000","last_peered":"2024-04-23T01:52:29.359805+0000","last_clean":"2024-04-23T01:52:29.359805+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:52:29.359805+0000","last_undegraded":"2024-04-23T01:52:29.359805+0000","last_fullsized":"2024-04-23T01:52:29.359805+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.374210+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:52:29.374210+0000","last_peered":"2024-04-23T01:52:29.374210+0000","last_clean":"2024-04-23T01:52:29.374210+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:52:29.374210+0000","last_undegraded":"2024-04-23T01:52:29.374210+0000","last_fullsized":"2024-04-23T01:52:29.374210+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"48'383","reported_seq":396,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:06.459642+0000","last_change":"2024-04-23T01:52:28.481837+0000","last_active":"2024-04-23T01:53:06.459642+0000","last_peered":"2024-04-23T01:53:06.459642+0000","last_clean":"2024-04-23T01:53:06.459642+0000","last_became_active":"2024-04-23T01:52:28.472032+0000","last_became_peered":"2024-04-23T01:52:28.472032+0000","last_unstale":"2024-04-23T01:53:06.459642+0000","last_undegraded":"2024-04-23T01:53:06.459642+0000","last_fullsized":"2024-04-23T01:53:06.459642+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":384,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":102,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.628985+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:52:29.628985+0000","last_peered":"2024-04-23T01:52:29.628985+0000","last_clean":"2024-04-23T01:52:29.628985+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:52:29.628985+0000","last_undegraded":"2024-04-23T01:52:29.628985+0000","last_fullsized":"2024-04-23T01:52:29.628985+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"48'391","reported_seq":404,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:05.821596+0000","last_change":"2024-04-23T01:52:28.529295+0000","last_active":"2024-04-23T01:53:05.821596+0000","last_peered":"2024-04-23T01:53:05.821596+0000","last_clean":"2024-04-23T01:53:05.821596+0000","last_became_active":"2024-04-23T01:52:28.523577+0000","last_became_peered":"2024-04-23T01:52:28.523577+0000","last_unstale":"2024-04-23T01:53:05.821596+0000","last_undegraded":"2024-04-23T01:53:05.821596+0000","last_fullsized":"2024-04-23T01:53:05.821596+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3211264,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":392,"num_write_kb":3136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.368290+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:52:29.368290+0000","last_peered":"2024-04-23T01:52:29.368290+0000","last_clean":"2024-04-23T01:52:29.368290+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:52:29.368290+0000","last_undegraded":"2024-04-23T01:52:29.368290+0000","last_fullsized":"2024-04-23T01:52:29.368290+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"48'303","reported_seq":316,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:05.592394+0000","last_change":"2024-04-23T01:52:28.456738+0000","last_active":"2024-04-23T01:53:05.592394+0000","last_peered":"2024-04-23T01:53:05.592394+0000","last_clean":"2024-04-23T01:53:05.592394+0000","last_became_active":"2024-04-23T01:52:28.432791+0000","last_became_peered":"2024-04-23T01:52:28.432791+0000","last_unstale":"2024-04-23T01:53:05.592394+0000","last_undegraded":"2024-04-23T01:53:05.592394+0000","last_fullsized":"2024-04-23T01:53:05.592394+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490368,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":102,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.623888+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:52:29.623888+0000","last_peered":"2024-04-23T01:52:29.623888+0000","last_clean":"2024-04-23T01:52:29.623888+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:52:29.623888+0000","last_undegraded":"2024-04-23T01:52:29.623888+0000","last_fullsized":"2024-04-23T01:52:29.623888+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"48'479","reported_seq":492,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:06.380542+0000","last_change":"2024-04-23T01:52:28.427753+0000","last_active":"2024-04-23T01:53:06.380542+0000","last_peered":"2024-04-23T01:53:06.380542+0000","last_clean":"2024-04-23T01:53:06.380542+0000","last_became_active":"2024-04-23T01:52:28.423538+0000","last_became_peered":"2024-04-23T01:52:28.423538+0000","last_unstale":"2024-04-23T01:53:06.380542+0000","last_undegraded":"2024-04-23T01:53:06.380542+0000","last_fullsized":"2024-04-23T01:53:06.380542+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":480,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":108,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.278611+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:52:29.278611+0000","last_peered":"2024-04-23T01:52:29.278611+0000","last_clean":"2024-04-23T01:52:29.278611+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:52:29.278611+0000","last_undegraded":"2024-04-23T01:52:29.278611+0000","last_fullsized":"2024-04-23T01:52:29.278611+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"48'311","reported_seq":324,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:05.989691+0000","last_change":"2024-04-23T01:52:28.503432+0000","last_active":"2024-04-23T01:53:05.989691+0000","last_peered":"2024-04-23T01:53:05.989691+0000","last_clean":"2024-04-23T01:53:05.989691+0000","last_became_active":"2024-04-23T01:52:28.498953+0000","last_became_peered":"2024-04-23T01:52:28.498953+0000","last_unstale":"2024-04-23T01:53:05.989691+0000","last_undegraded":"2024-04-23T01:53:05.989691+0000","last_fullsized":"2024-04-23T01:53:05.989691+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2555904,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":2496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":104,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.389394+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:52:29.389394+0000","last_peered":"2024-04-23T01:52:29.389394+0000","last_clean":"2024-04-23T01:52:29.389394+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:52:29.389394+0000","last_undegraded":"2024-04-23T01:52:29.389394+0000","last_fullsized":"2024-04-23T01:52:29.389394+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"48'298","reported_seq":311,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:06.671092+0000","last_change":"2024-04-23T01:52:28.486992+0000","last_active":"2024-04-23T01:53:06.671092+0000","last_peered":"2024-04-23T01:53:06.671092+0000","last_clean":"2024-04-23T01:53:06.671092+0000","last_became_active":"2024-04-23T01:52:28.482639+0000","last_became_peered":"2024-04-23T01:52:28.482639+0000","last_unstale":"2024-04-23T01:53:06.671092+0000","last_undegraded":"2024-04-23T01:53:06.671092+0000","last_fullsized":"2024-04-23T01:53:06.671092+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2449408,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":299,"num_write_kb":2392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":108,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.297789+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:52:29.297789+0000","last_peered":"2024-04-23T01:52:29.297789+0000","last_clean":"2024-04-23T01:52:29.297789+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:52:29.297789+0000","last_undegraded":"2024-04-23T01:52:29.297789+0000","last_fullsized":"2024-04-23T01:52:29.297789+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"48'263","reported_seq":276,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:05.531113+0000","last_change":"2024-04-23T01:52:28.453502+0000","last_active":"2024-04-23T01:53:05.531113+0000","last_peered":"2024-04-23T01:53:05.531113+0000","last_clean":"2024-04-23T01:53:05.531113+0000","last_became_active":"2024-04-23T01:52:28.448199+0000","last_became_peered":"2024-04-23T01:52:28.448199+0000","last_unstale":"2024-04-23T01:53:05.531113+0000","last_undegraded":"2024-04-23T01:53:05.531113+0000","last_fullsized":"2024-04-23T01:53:05.531113+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162688,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":264,"num_write_kb":2112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.366780+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:52:29.366780+0000","last_peered":"2024-04-23T01:52:29.366780+0000","last_clean":"2024-04-23T01:52:29.366780+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:52:29.366780+0000","last_undegraded":"2024-04-23T01:52:29.366780+0000","last_fullsized":"2024-04-23T01:52:29.366780+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":45064192,"num_objects":689,"num_object_clones":0,"num_object_copies":2067,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5501,"num_write_kb":44008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":67559424,"data_stored":135118848,"data_compressed":6973632,"data_compressed_allocated":67559424,"data_compressed_original":135118848,"omap_allocated":0,"internal_metadata":0},"log_size":5485,"ondisk_log_size":5485,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706143,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":105920,"kb_used_data":20792,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94265920,"statfs":{"total":96636764160,"available":96528302080,"internally_reserved":0,"allocated":21291008,"data_stored":42367996,"data_compressed":2178569,"data_compressed_allocated":20856832,"data_compressed_original":42205184,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706143,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":97880,"kb_used_data":12752,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94273960,"statfs":{"total":96636764160,"available":96536535040,"internally_reserved":0,"allocated":13058048,"data_stored":25902076,"data_compressed":1328781,"data_compressed_allocated":12623872,"data_compressed_original":25739264,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706143,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":111172,"kb_used_data":17724,"kb_used_omap":0,"kb_used_meta":93440,"kb_avail":94260668,"statfs":{"total":96636764160,"available":96522924032,"internally_reserved":0,"allocated":18149376,"data_stored":35617244,"data_compressed":1830736,"data_compressed_allocated":17735680,"data_compressed_original":35471360,"omap_allocated":0,"internal_metadata":95682560},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706143,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":109284,"kb_used_data":15836,"kb_used_omap":0,"kb_used_meta":93440,"kb_avail":94262556,"statfs":{"total":96636764160,"available":96524857344,"internally_reserved":0,"allocated":16216064,"data_stored":32218108,"data_compressed":1654764,"data_compressed_allocated":15781888,"data_compressed_original":32055296,"omap_allocated":0,"internal_metadata":95682560},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15835136,"data_stored":31670272,"data_compressed":1634540,"data_compressed_allocated":15835136,"data_compressed_original":31670272,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":17944576,"data_stored":35889152,"data_compressed":1852309,"data_compressed_allocated":17944576,"data_compressed_original":35889152,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12718080,"data_stored":25436160,"data_compressed":1312787,"data_compressed_allocated":12718080,"data_compressed_original":25436160,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":21061632,"data_stored":42123264,"data_compressed":2173996,"data_compressed_allocated":21061632,"data_compressed_original":42123264,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:53:12.343 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 43 16 6307 6291 1.14289 1.16406 0.143603 0.109219 2024-04-23T01:53:13.075 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:53:13.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:13.193+0000 7f1455d5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:13.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:13.199+0000 7f1455d5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:13.343 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 44 16 6453 6437 1.14283 1.14062 0.227212 0.109179 2024-04-23T01:53:14.343 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 45 16 6596 6580 1.14226 1.11719 0.0616589 0.109284 2024-04-23T01:53:15.343 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 46 16 6734 6718 1.14086 1.07812 0.114922 0.109416 2024-04-23T01:53:15.407 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:53:15.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:15.524+0000 7f0a9a46d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:15.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:15.531+0000 7f0a9a46d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:16.343 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 47 16 6876 6860 1.14019 1.10938 0.152835 0.109485 2024-04-23T01:53:17.222 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:53:17.222 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:53:17.223 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:53:17.223 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:53:17.343 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 48 16 7016 7000 1.13922 1.09375 0.108707 0.109578 2024-04-23T01:53:17.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:17.342+0000 7fb4780ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:17.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:17.352+0000 7fb4780ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:17.375 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:53:17.376 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:53:17.376 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:53:17.376 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:53:17.376 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:53:17.376 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:53:17.376 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:53:17.376 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:53:17.376 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:53:17.376 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:53:17.376 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:53:17.376 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:53:17.379 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:53:17.379 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:53:17.380 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:53:17.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:17.491+0000 7fc5c52ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:17.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:17.501+0000 7fc5c52ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:17.550 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:53:17.550 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:53:17.550 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:53:17.550 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:53:17.550 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:53:17.550 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:53:17.550 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:53:17.550 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:53:17.550 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:53:17.550 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:53:17.551 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:53:17.551 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:53:17.553 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:53:17.553 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:53:17.553 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:53:17.688 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:53:17.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:17.691+0000 7f46ff15d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:17.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:17.706+0000 7f46ff15d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:17.728 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:53:17.728 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:53:17.729 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:53:17.729 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:53:17.729 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:53:17.729 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:53:17.729 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:53:17.729 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:53:17.729 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:53:17.729 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:53:17.729 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:53:17.729 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:53:17.733 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:53:17.733 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:53:17.733 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:53:17.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:17.813+0000 7fa8490ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:17.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:17.822+0000 7fa8490ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:17.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:17.933+0000 7fe749f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:17.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:17.946+0000 7fe749f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:17.986 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:53:17.987 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:53:17.987 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:53:17.987 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:53:17.987 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:53:17.987 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:53:17.987 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:53:17.987 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:53:17.987 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:53:17.987 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:53:17.987 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:53:17.987 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:53:17.990 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:53:17.991 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:53:18.343 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 49 16 7144 7128 1.13638 1 0.18679 0.109767 2024-04-23T01:53:19.343 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 50 16 7281 7265 1.13506 1.07031 0.189372 0.109948 2024-04-23T01:53:20.075 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:53:20.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:20.212+0000 7f4edcaad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:20.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:20.219+0000 7f4edcaad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:20.343 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 51 16 7425 7409 1.13486 1.125 0.132935 0.109984 2024-04-23T01:53:21.344 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 52 16 7570 7554 1.13482 1.13281 0.0629514 0.110016 2024-04-23T01:53:22.344 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 53 16 7713 7697 1.13448 1.11719 0.0479399 0.110019 2024-04-23T01:53:22.476 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:53:22.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:22.602+0000 7f102ae6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:22.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:22.610+0000 7f102ae6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:22.992 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:53:22.992 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:53:22.992 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:53:22.993 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:53:23.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:23.123+0000 7f2755020640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:23.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:23.137+0000 7f2755020640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:23.193 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:53:23.193 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:53:23.193 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:53:23.193 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:53:23.193 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:53:23.193 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:53:23.193 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:53:23.194 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:53:23.194 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:53:23.194 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:53:23.194 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:53:23.194 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:53:23.197 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:53:23.197 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:53:23.198 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:53:23.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:23.334+0000 7fa1846ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:23.343 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 54 16 7858 7842 1.13445 1.13281 0.168152 0.110077 2024-04-23T01:53:23.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:23.345+0000 7fa1846ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:23.457 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:53:23.457 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:53:23.457 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:53:23.457 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:53:23.457 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:53:23.457 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:53:23.457 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:53:23.457 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:53:23.457 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:53:23.457 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:53:23.458 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:53:23.458 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:53:23.460 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:53:23.460 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:53:23.460 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:53:23.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:23.578+0000 7f419446d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:23.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:23.589+0000 7f419446d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:23.605 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:53:23.605 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:53:23.605 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:53:23.605 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:53:23.605 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:53:23.605 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:53:23.605 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:53:23.606 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:53:23.606 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:53:23.606 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:53:23.606 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:53:23.606 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:53:23.608 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:53:23.608 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:53:23.609 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:53:23.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:23.719+0000 7f63e68ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:23.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:23.734+0000 7f63e68ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:23.795 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:53:23.796 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:53:23.796 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:53:23.796 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:53:23.796 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:53:23.796 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:53:23.796 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:53:23.796 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:53:23.796 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:53:23.796 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:53:23.796 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:53:23.796 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:53:23.799 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:53:23.799 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:53:24.344 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 55 16 7981 7965 1.13129 0.960938 0.0840426 0.11031 2024-04-23T01:53:24.868 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:53:24.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:24.985+0000 7f3d1306d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:24.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:24.992+0000 7f3d1306d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:25.344 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 56 16 8113 8097 1.12951 1.03125 0.159616 0.110511 2024-04-23T01:53:26.344 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 57 16 8245 8229 1.12778 1.03125 0.109148 0.110688 2024-04-23T01:53:27.228 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:53:27.347 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 58 16 8379 8363 1.12638 1.04688 0.062676 0.110849 2024-04-23T01:53:27.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:27.375+0000 7efd3a06d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:27.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:27.381+0000 7efd3a06d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:28.344 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 59 16 8501 8485 1.12345 0.953125 0.180784 0.111101 2024-04-23T01:53:28.800 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:53:28.800 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:53:28.800 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:53:28.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:28.912+0000 7f00cb66d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:28.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:28.920+0000 7f00cb66d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:29.124 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:53:29.125 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:53:29.138 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-04-23T01:53:27.644825+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":68461088,"num_objects":1039,"num_object_clones":0,"num_object_copies":3117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":8422,"num_write_kb":68174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8366,"ondisk_log_size":8366,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":457600,"kb_used_data":100448,"kb_used_omap":0,"kb_used_meta":357120,"kb_avail":377029760,"statfs":{"total":386547056640,"available":386078474240,"internally_reserved":0,"allocated":102858752,"data_stored":204393936,"data_compressed":10518978,"data_compressed_allocated":101142528,"data_compressed_original":203759616,"omap_allocated":0,"internal_metadata":365690880},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17096704,"num_objects":261,"num_object_clones":0,"num_object_copies":783,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2087,"num_write_kb":16696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002141"},"pg_stats":[{"pgid":"7.f","version":"48'487","reported_seq":500,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:26.613962+0000","last_change":"2024-04-23T01:52:28.525229+0000","last_active":"2024-04-23T01:53:26.613962+0000","last_peered":"2024-04-23T01:53:26.613962+0000","last_clean":"2024-04-23T01:53:26.613962+0000","last_became_active":"2024-04-23T01:52:28.518971+0000","last_became_peered":"2024-04-23T01:52:28.518971+0000","last_unstale":"2024-04-23T01:53:26.613962+0000","last_undegraded":"2024-04-23T01:53:26.613962+0000","last_fullsized":"2024-04-23T01:53:26.613962+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3997696,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":488,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"48'527","reported_seq":540,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:26.060615+0000","last_change":"2024-04-23T01:52:28.530871+0000","last_active":"2024-04-23T01:53:26.060615+0000","last_peered":"2024-04-23T01:53:26.060615+0000","last_clean":"2024-04-23T01:53:26.060615+0000","last_became_active":"2024-04-23T01:52:28.521696+0000","last_became_peered":"2024-04-23T01:52:28.521696+0000","last_unstale":"2024-04-23T01:53:26.060615+0000","last_undegraded":"2024-04-23T01:53:26.060615+0000","last_fullsized":"2024-04-23T01:53:26.060615+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":528,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"48'519","reported_seq":532,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:26.185938+0000","last_change":"2024-04-23T01:52:28.407738+0000","last_active":"2024-04-23T01:53:26.185938+0000","last_peered":"2024-04-23T01:53:26.185938+0000","last_clean":"2024-04-23T01:53:26.185938+0000","last_became_active":"2024-04-23T01:52:28.400739+0000","last_became_peered":"2024-04-23T01:52:28.400739+0000","last_unstale":"2024-04-23T01:53:26.185938+0000","last_undegraded":"2024-04-23T01:53:26.185938+0000","last_fullsized":"2024-04-23T01:53:26.185938+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259840,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":520,"num_write_kb":4160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"48'468","reported_seq":481,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:26.693714+0000","last_change":"2024-04-23T01:52:28.435954+0000","last_active":"2024-04-23T01:53:26.693714+0000","last_peered":"2024-04-23T01:53:26.693714+0000","last_clean":"2024-04-23T01:53:26.693714+0000","last_became_active":"2024-04-23T01:52:28.430557+0000","last_became_peered":"2024-04-23T01:52:28.430557+0000","last_unstale":"2024-04-23T01:53:26.693714+0000","last_undegraded":"2024-04-23T01:53:26.693714+0000","last_fullsized":"2024-04-23T01:53:26.693714+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842048,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":469,"num_write_kb":3752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"48'440","reported_seq":453,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:24.436085+0000","last_change":"2024-04-23T01:52:28.452034+0000","last_active":"2024-04-23T01:53:24.436085+0000","last_peered":"2024-04-23T01:53:24.436085+0000","last_clean":"2024-04-23T01:53:24.436085+0000","last_became_active":"2024-04-23T01:52:28.430826+0000","last_became_peered":"2024-04-23T01:52:28.430826+0000","last_unstale":"2024-04-23T01:53:24.436085+0000","last_undegraded":"2024-04-23T01:53:24.436085+0000","last_fullsized":"2024-04-23T01:53:24.436085+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3612672,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":441,"num_write_kb":3528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"48'615","reported_seq":628,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:26.342985+0000","last_change":"2024-04-23T01:52:28.484337+0000","last_active":"2024-04-23T01:53:26.342985+0000","last_peered":"2024-04-23T01:53:26.342985+0000","last_clean":"2024-04-23T01:53:26.342985+0000","last_became_active":"2024-04-23T01:52:28.476566+0000","last_became_peered":"2024-04-23T01:52:28.476566+0000","last_unstale":"2024-04-23T01:53:26.342985+0000","last_undegraded":"2024-04-23T01:53:26.342985+0000","last_fullsized":"2024-04-23T01:53:26.342985+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5046272,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":616,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"48'503","reported_seq":516,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:26.654953+0000","last_change":"2024-04-23T01:52:28.418531+0000","last_active":"2024-04-23T01:53:26.654953+0000","last_peered":"2024-04-23T01:53:26.654953+0000","last_clean":"2024-04-23T01:53:26.654953+0000","last_became_active":"2024-04-23T01:52:28.412725+0000","last_became_peered":"2024-04-23T01:52:28.412725+0000","last_unstale":"2024-04-23T01:53:26.654953+0000","last_undegraded":"2024-04-23T01:53:26.654953+0000","last_fullsized":"2024-04-23T01:53:26.654953+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4128768,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":504,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"48'551","reported_seq":564,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:26.231163+0000","last_change":"2024-04-23T01:52:28.498385+0000","last_active":"2024-04-23T01:53:26.231163+0000","last_peered":"2024-04-23T01:53:26.231163+0000","last_clean":"2024-04-23T01:53:26.231163+0000","last_became_active":"2024-04-23T01:52:28.487852+0000","last_became_peered":"2024-04-23T01:52:28.487852+0000","last_unstale":"2024-04-23T01:53:26.231163+0000","last_undegraded":"2024-04-23T01:53:26.231163+0000","last_fullsized":"2024-04-23T01:53:26.231163+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4521984,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":552,"num_write_kb":4416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"48'567","reported_seq":580,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:26.178874+0000","last_change":"2024-04-23T01:52:28.536598+0000","last_active":"2024-04-23T01:53:26.178874+0000","last_peered":"2024-04-23T01:53:26.178874+0000","last_clean":"2024-04-23T01:53:26.178874+0000","last_became_active":"2024-04-23T01:52:28.530283+0000","last_became_peered":"2024-04-23T01:52:28.530283+0000","last_unstale":"2024-04-23T01:53:26.178874+0000","last_undegraded":"2024-04-23T01:53:26.178874+0000","last_fullsized":"2024-04-23T01:53:26.178874+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":568,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":237,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.359805+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:52:29.359805+0000","last_peered":"2024-04-23T01:52:29.359805+0000","last_clean":"2024-04-23T01:52:29.359805+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:52:29.359805+0000","last_undegraded":"2024-04-23T01:52:29.359805+0000","last_fullsized":"2024-04-23T01:52:29.359805+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.374210+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:52:29.374210+0000","last_peered":"2024-04-23T01:52:29.374210+0000","last_clean":"2024-04-23T01:52:29.374210+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:52:29.374210+0000","last_undegraded":"2024-04-23T01:52:29.374210+0000","last_fullsized":"2024-04-23T01:52:29.374210+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"48'599","reported_seq":612,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:26.474677+0000","last_change":"2024-04-23T01:52:28.481837+0000","last_active":"2024-04-23T01:53:26.474677+0000","last_peered":"2024-04-23T01:53:26.474677+0000","last_clean":"2024-04-23T01:53:26.474677+0000","last_became_active":"2024-04-23T01:52:28.472032+0000","last_became_peered":"2024-04-23T01:52:28.472032+0000","last_unstale":"2024-04-23T01:53:26.474677+0000","last_undegraded":"2024-04-23T01:53:26.474677+0000","last_fullsized":"2024-04-23T01:53:26.474677+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":102,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.628985+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:52:29.628985+0000","last_peered":"2024-04-23T01:52:29.628985+0000","last_clean":"2024-04-23T01:52:29.628985+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:52:29.628985+0000","last_undegraded":"2024-04-23T01:52:29.628985+0000","last_fullsized":"2024-04-23T01:52:29.628985+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"48'591","reported_seq":604,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:24.098089+0000","last_change":"2024-04-23T01:52:28.529295+0000","last_active":"2024-04-23T01:53:24.098089+0000","last_peered":"2024-04-23T01:53:24.098089+0000","last_clean":"2024-04-23T01:53:24.098089+0000","last_became_active":"2024-04-23T01:52:28.523577+0000","last_became_peered":"2024-04-23T01:52:28.523577+0000","last_unstale":"2024-04-23T01:53:24.098089+0000","last_undegraded":"2024-04-23T01:53:24.098089+0000","last_fullsized":"2024-04-23T01:53:24.098089+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":592,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.368290+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:52:29.368290+0000","last_peered":"2024-04-23T01:52:29.368290+0000","last_clean":"2024-04-23T01:52:29.368290+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:52:29.368290+0000","last_undegraded":"2024-04-23T01:52:29.368290+0000","last_fullsized":"2024-04-23T01:52:29.368290+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"48'455","reported_seq":468,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:26.359403+0000","last_change":"2024-04-23T01:52:28.456738+0000","last_active":"2024-04-23T01:53:26.359403+0000","last_peered":"2024-04-23T01:53:26.359403+0000","last_clean":"2024-04-23T01:53:26.359403+0000","last_became_active":"2024-04-23T01:52:28.432791+0000","last_became_peered":"2024-04-23T01:52:28.432791+0000","last_unstale":"2024-04-23T01:53:26.359403+0000","last_undegraded":"2024-04-23T01:53:26.359403+0000","last_fullsized":"2024-04-23T01:53:26.359403+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":456,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":102,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.623888+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:52:29.623888+0000","last_peered":"2024-04-23T01:52:29.623888+0000","last_clean":"2024-04-23T01:52:29.623888+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:52:29.623888+0000","last_undegraded":"2024-04-23T01:52:29.623888+0000","last_fullsized":"2024-04-23T01:52:29.623888+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"48'638","reported_seq":651,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:26.615013+0000","last_change":"2024-04-23T01:52:28.427753+0000","last_active":"2024-04-23T01:53:26.615013+0000","last_peered":"2024-04-23T01:53:26.615013+0000","last_clean":"2024-04-23T01:53:26.615013+0000","last_became_active":"2024-04-23T01:52:28.423538+0000","last_became_peered":"2024-04-23T01:52:28.423538+0000","last_unstale":"2024-04-23T01:53:26.615013+0000","last_undegraded":"2024-04-23T01:53:26.615013+0000","last_fullsized":"2024-04-23T01:53:26.615013+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5234688,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":639,"num_write_kb":5112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":108,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.278611+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:52:29.278611+0000","last_peered":"2024-04-23T01:52:29.278611+0000","last_clean":"2024-04-23T01:52:29.278611+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:52:29.278611+0000","last_undegraded":"2024-04-23T01:52:29.278611+0000","last_fullsized":"2024-04-23T01:52:29.278611+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"48'431","reported_seq":444,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:26.429416+0000","last_change":"2024-04-23T01:52:28.503432+0000","last_active":"2024-04-23T01:53:26.429416+0000","last_peered":"2024-04-23T01:53:26.429416+0000","last_clean":"2024-04-23T01:53:26.429416+0000","last_became_active":"2024-04-23T01:52:28.498953+0000","last_became_peered":"2024-04-23T01:52:28.498953+0000","last_unstale":"2024-04-23T01:53:26.429416+0000","last_undegraded":"2024-04-23T01:53:26.429416+0000","last_fullsized":"2024-04-23T01:53:26.429416+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3538944,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":432,"num_write_kb":3456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":104,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.389394+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:52:29.389394+0000","last_peered":"2024-04-23T01:52:29.389394+0000","last_clean":"2024-04-23T01:52:29.389394+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:52:29.389394+0000","last_undegraded":"2024-04-23T01:52:29.389394+0000","last_fullsized":"2024-04-23T01:52:29.389394+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"48'431","reported_seq":444,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:25.493632+0000","last_change":"2024-04-23T01:52:28.486992+0000","last_active":"2024-04-23T01:53:25.493632+0000","last_peered":"2024-04-23T01:53:25.493632+0000","last_clean":"2024-04-23T01:53:25.493632+0000","last_became_active":"2024-04-23T01:52:28.482639+0000","last_became_peered":"2024-04-23T01:52:28.482639+0000","last_unstale":"2024-04-23T01:53:25.493632+0000","last_undegraded":"2024-04-23T01:53:25.493632+0000","last_fullsized":"2024-04-23T01:53:25.493632+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3538944,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":432,"num_write_kb":3456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":108,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.297789+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:52:29.297789+0000","last_peered":"2024-04-23T01:52:29.297789+0000","last_clean":"2024-04-23T01:52:29.297789+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:52:29.297789+0000","last_undegraded":"2024-04-23T01:52:29.297789+0000","last_fullsized":"2024-04-23T01:52:29.297789+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"48'447","reported_seq":460,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:26.546817+0000","last_change":"2024-04-23T01:52:28.453502+0000","last_active":"2024-04-23T01:53:26.546817+0000","last_peered":"2024-04-23T01:53:26.546817+0000","last_clean":"2024-04-23T01:53:26.546817+0000","last_became_active":"2024-04-23T01:52:28.448199+0000","last_became_peered":"2024-04-23T01:52:28.448199+0000","last_unstale":"2024-04-23T01:53:26.546817+0000","last_undegraded":"2024-04-23T01:53:26.546817+0000","last_fullsized":"2024-04-23T01:53:26.546817+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":448,"num_write_kb":3584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.366780+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:52:29.366780+0000","last_peered":"2024-04-23T01:52:29.366780+0000","last_clean":"2024-04-23T01:52:29.366780+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:52:29.366780+0000","last_undegraded":"2024-04-23T01:52:29.366780+0000","last_fullsized":"2024-04-23T01:52:29.366780+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":67870720,"num_objects":1037,"num_object_clones":0,"num_object_copies":3111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8285,"num_write_kb":66280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":101777408,"data_stored":203554816,"data_compressed":10507374,"data_compressed_allocated":101777408,"data_compressed_original":203554816,"omap_allocated":0,"internal_metadata":0},"log_size":8269,"ondisk_log_size":8269,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706163,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":128812,"kb_used_data":31204,"kb_used_omap":0,"kb_used_meta":97600,"kb_avail":94243028,"statfs":{"total":96636764160,"available":96504860672,"internally_reserved":0,"allocated":31952896,"data_stored":63691772,"data_compressed":3279638,"data_compressed_allocated":31518720,"data_compressed_original":63528960,"omap_allocated":0,"internal_metadata":99942400},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706163,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":108376,"kb_used_data":19088,"kb_used_omap":0,"kb_used_meta":89280,"kb_avail":94263464,"statfs":{"total":96636764160,"available":96525787136,"internally_reserved":0,"allocated":19546112,"data_stored":38878204,"data_compressed":1998813,"data_compressed_allocated":19111936,"data_compressed_original":38715392,"omap_allocated":0,"internal_metadata":91422720},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706163,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":111884,"kb_used_data":26756,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94259956,"statfs":{"total":96636764160,"available":96522194944,"internally_reserved":0,"allocated":27398144,"data_stored":54114780,"data_compressed":2785870,"data_compressed_allocated":26984448,"data_compressed_original":53968896,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706163,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":108528,"kb_used_data":23400,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94263312,"statfs":{"total":96636764160,"available":96525631488,"internally_reserved":0,"allocated":23961600,"data_stored":47709180,"data_compressed":2454657,"data_compressed_allocated":23527424,"data_compressed_original":47546368,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":23584768,"data_stored":47169536,"data_compressed":2434856,"data_compressed_allocated":23584768,"data_compressed_original":47169536,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":27193344,"data_stored":54386688,"data_compressed":2807443,"data_compressed_allocated":27193344,"data_compressed_original":54386688,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19222528,"data_stored":38445056,"data_compressed":1984511,"data_compressed_allocated":19222528,"data_compressed_original":38445056,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":31776768,"data_stored":63553536,"data_compressed":3280564,"data_compressed_allocated":31776768,"data_compressed_original":63553536,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:53:29.344 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:53:29.344566+0000 min lat: 0.0154323 max lat: 0.370563 avg lat: 0.111212 2024-04-23T01:53:29.344 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:53:29.344 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 60 16 8638 8622 1.12256 1.07031 0.125753 0.111212 2024-04-23T01:53:29.616 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:53:29.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:29.739+0000 7f7f8c62d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:29.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:29.746+0000 7f7f8c62d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:30.344 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 61 16 8775 8759 1.1217 1.07031 0.0724109 0.11131 2024-04-23T01:53:31.344 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 62 16 8923 8907 1.12226 1.15625 0.0910733 0.111226 2024-04-23T01:53:32.039 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:53:32.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:32.160+0000 7fef1835d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:32.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:32.166+0000 7fef1835d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:32.344 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 63 16 9056 9040 1.12093 1.03906 0.171242 0.111356 2024-04-23T01:53:33.345 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 64 16 9186 9170 1.11929 1.01562 0.121732 0.111524 2024-04-23T01:53:34.141 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:53:34.141 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:53:34.141 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:53:34.141 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:53:34.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:34.258+0000 7ffb30f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:34.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:34.268+0000 7ffb30f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:34.295 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:53:34.295 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:53:34.295 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:53:34.295 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:53:34.295 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:53:34.295 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:53:34.295 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:53:34.295 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:53:34.295 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:53:34.295 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:53:34.295 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:53:34.296 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:53:34.298 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:53:34.299 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:53:34.299 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:53:34.345 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 65 16 9330 9314 1.11937 1.125 0.181043 0.111564 2024-04-23T01:53:34.413 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:53:34.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:34.409+0000 7f0c9ce0f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:34.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:34.419+0000 7f0c9ce0f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:34.463 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:53:34.463 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:53:34.463 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:53:34.463 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:53:34.463 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:53:34.463 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:53:34.463 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:53:34.463 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:53:34.463 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:53:34.463 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:53:34.463 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:53:34.464 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:53:34.466 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:53:34.467 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:53:34.467 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:53:34.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:34.537+0000 7ffa4686d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:34.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:34.543+0000 7ffa4686d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:34.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:34.643+0000 7fba1a8ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:34.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:34.653+0000 7fba1a8ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:34.691 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:53:34.691 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:53:34.691 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:53:34.691 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:53:34.691 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:53:34.691 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:53:34.692 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:53:34.692 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:53:34.692 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:53:34.692 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:53:34.692 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:53:34.692 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:53:34.695 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:53:34.695 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:53:34.695 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:53:34.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:34.896+0000 7f8318aed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:34.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:34.907+0000 7f8318aed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:34.944 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:53:34.944 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:53:34.944 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:53:34.944 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:53:34.944 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:53:34.944 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:53:34.944 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:53:34.944 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:53:34.945 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:53:34.945 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:53:34.945 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:53:34.945 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:53:34.947 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:53:34.947 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:53:35.345 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 66 16 9461 9445 1.11792 1.02344 0.0655311 0.111715 2024-04-23T01:53:36.346 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 67 16 9591 9575 1.11639 1.01562 0.0351134 0.111799 2024-04-23T01:53:36.714 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:53:36.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:36.824+0000 7f5e8c42d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:36.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:36.831+0000 7f5e8c42d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:37.345 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 68 16 9725 9709 1.11537 1.04688 0.0976928 0.111967 2024-04-23T01:53:38.345 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 69 16 9855 9839 1.11392 1.01562 0.12302 0.112066 2024-04-23T01:53:39.002 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:53:39.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:39.112+0000 7f13e3cad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:39.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:39.118+0000 7f13e3cad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:39.345 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 70 16 9985 9969 1.11251 1.01562 0.165111 0.112234 2024-04-23T01:53:39.948 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:53:39.949 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:53:39.949 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:53:40.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:40.098+0000 7f270af9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:40.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:40.110+0000 7f270af9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:40.339 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:53:40.339 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:53:40.345 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 71 16 10111 10095 1.11071 0.984375 0.0515928 0.112377 2024-04-23T01:53:40.352 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2024-04-23T01:53:39.647175+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":79413792,"num_objects":1206,"num_object_clones":0,"num_object_copies":3618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":9759,"num_write_kb":78870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9703,"ondisk_log_size":9703,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":473780,"kb_used_data":116628,"kb_used_omap":0,"kb_used_meta":357120,"kb_avail":377013580,"statfs":{"total":386547056640,"available":386061905920,"internally_reserved":0,"allocated":119427072,"data_stored":237530576,"data_compressed":12230013,"data_compressed_allocated":117710848,"data_compressed_original":236896256,"omap_allocated":0,"internal_metadata":365690880},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10952704,"num_objects":167,"num_object_clones":0,"num_object_copies":501,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1337,"num_write_kb":10696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002349"},"pg_stats":[{"pgid":"7.f","version":"48'559","reported_seq":572,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:36.557724+0000","last_change":"2024-04-23T01:52:28.525229+0000","last_active":"2024-04-23T01:53:36.557724+0000","last_peered":"2024-04-23T01:53:36.557724+0000","last_clean":"2024-04-23T01:53:36.557724+0000","last_became_active":"2024-04-23T01:52:28.518971+0000","last_became_peered":"2024-04-23T01:52:28.518971+0000","last_unstale":"2024-04-23T01:53:36.557724+0000","last_undegraded":"2024-04-23T01:53:36.557724+0000","last_fullsized":"2024-04-23T01:53:36.557724+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4587520,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":560,"num_write_kb":4480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"48'624","reported_seq":637,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:36.254644+0000","last_change":"2024-04-23T01:52:28.530871+0000","last_active":"2024-04-23T01:53:36.254644+0000","last_peered":"2024-04-23T01:53:36.254644+0000","last_clean":"2024-04-23T01:53:36.254644+0000","last_became_active":"2024-04-23T01:52:28.521696+0000","last_became_peered":"2024-04-23T01:52:28.521696+0000","last_unstale":"2024-04-23T01:53:36.254644+0000","last_undegraded":"2024-04-23T01:53:36.254644+0000","last_fullsized":"2024-04-23T01:53:36.254644+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5120000,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":625,"num_write_kb":5000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"48'583","reported_seq":596,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:34.268208+0000","last_change":"2024-04-23T01:52:28.407738+0000","last_active":"2024-04-23T01:53:34.268208+0000","last_peered":"2024-04-23T01:53:34.268208+0000","last_clean":"2024-04-23T01:53:34.268208+0000","last_became_active":"2024-04-23T01:52:28.400739+0000","last_became_peered":"2024-04-23T01:52:28.400739+0000","last_unstale":"2024-04-23T01:53:34.268208+0000","last_undegraded":"2024-04-23T01:53:34.268208+0000","last_fullsized":"2024-04-23T01:53:34.268208+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4784128,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":584,"num_write_kb":4672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"48'535","reported_seq":548,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:34.922274+0000","last_change":"2024-04-23T01:52:28.435954+0000","last_active":"2024-04-23T01:53:34.922274+0000","last_peered":"2024-04-23T01:53:34.922274+0000","last_clean":"2024-04-23T01:53:34.922274+0000","last_became_active":"2024-04-23T01:52:28.430557+0000","last_became_peered":"2024-04-23T01:52:28.430557+0000","last_unstale":"2024-04-23T01:53:34.922274+0000","last_undegraded":"2024-04-23T01:53:34.922274+0000","last_fullsized":"2024-04-23T01:53:34.922274+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4390912,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":536,"num_write_kb":4288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"48'495","reported_seq":508,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:35.838868+0000","last_change":"2024-04-23T01:52:28.452034+0000","last_active":"2024-04-23T01:53:35.838868+0000","last_peered":"2024-04-23T01:53:35.838868+0000","last_clean":"2024-04-23T01:53:35.838868+0000","last_became_active":"2024-04-23T01:52:28.430826+0000","last_became_peered":"2024-04-23T01:52:28.430826+0000","last_unstale":"2024-04-23T01:53:35.838868+0000","last_undegraded":"2024-04-23T01:53:35.838868+0000","last_fullsized":"2024-04-23T01:53:35.838868+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4063232,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":496,"num_write_kb":3968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"48'719","reported_seq":732,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:36.377290+0000","last_change":"2024-04-23T01:52:28.484337+0000","last_active":"2024-04-23T01:53:36.377290+0000","last_peered":"2024-04-23T01:53:36.377290+0000","last_clean":"2024-04-23T01:53:36.377290+0000","last_became_active":"2024-04-23T01:52:28.476566+0000","last_became_peered":"2024-04-23T01:52:28.476566+0000","last_unstale":"2024-04-23T01:53:36.377290+0000","last_undegraded":"2024-04-23T01:53:36.377290+0000","last_fullsized":"2024-04-23T01:53:36.377290+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":720,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"48'551","reported_seq":564,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:36.449295+0000","last_change":"2024-04-23T01:52:28.418531+0000","last_active":"2024-04-23T01:53:36.449295+0000","last_peered":"2024-04-23T01:53:36.449295+0000","last_clean":"2024-04-23T01:53:36.449295+0000","last_became_active":"2024-04-23T01:52:28.412725+0000","last_became_peered":"2024-04-23T01:52:28.412725+0000","last_unstale":"2024-04-23T01:53:36.449295+0000","last_undegraded":"2024-04-23T01:53:36.449295+0000","last_fullsized":"2024-04-23T01:53:36.449295+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4521984,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":552,"num_write_kb":4416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"48'599","reported_seq":612,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:36.041260+0000","last_change":"2024-04-23T01:52:28.498385+0000","last_active":"2024-04-23T01:53:36.041260+0000","last_peered":"2024-04-23T01:53:36.041260+0000","last_clean":"2024-04-23T01:53:36.041260+0000","last_became_active":"2024-04-23T01:52:28.487852+0000","last_became_peered":"2024-04-23T01:52:28.487852+0000","last_unstale":"2024-04-23T01:53:36.041260+0000","last_undegraded":"2024-04-23T01:53:36.041260+0000","last_fullsized":"2024-04-23T01:53:36.041260+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"48'631","reported_seq":644,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:36.317838+0000","last_change":"2024-04-23T01:52:28.536598+0000","last_active":"2024-04-23T01:53:36.317838+0000","last_peered":"2024-04-23T01:53:36.317838+0000","last_clean":"2024-04-23T01:53:36.317838+0000","last_became_active":"2024-04-23T01:52:28.530283+0000","last_became_peered":"2024-04-23T01:52:28.530283+0000","last_unstale":"2024-04-23T01:53:36.317838+0000","last_undegraded":"2024-04-23T01:53:36.317838+0000","last_fullsized":"2024-04-23T01:53:36.317838+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":5056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":237,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.359805+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:52:29.359805+0000","last_peered":"2024-04-23T01:52:29.359805+0000","last_clean":"2024-04-23T01:52:29.359805+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:52:29.359805+0000","last_undegraded":"2024-04-23T01:52:29.359805+0000","last_fullsized":"2024-04-23T01:52:29.359805+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.374210+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:52:29.374210+0000","last_peered":"2024-04-23T01:52:29.374210+0000","last_clean":"2024-04-23T01:52:29.374210+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:52:29.374210+0000","last_undegraded":"2024-04-23T01:52:29.374210+0000","last_fullsized":"2024-04-23T01:52:29.374210+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"48'719","reported_seq":732,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:36.349929+0000","last_change":"2024-04-23T01:52:28.481837+0000","last_active":"2024-04-23T01:53:36.349929+0000","last_peered":"2024-04-23T01:53:36.349929+0000","last_clean":"2024-04-23T01:53:36.349929+0000","last_became_active":"2024-04-23T01:52:28.472032+0000","last_became_peered":"2024-04-23T01:52:28.472032+0000","last_unstale":"2024-04-23T01:53:36.349929+0000","last_undegraded":"2024-04-23T01:53:36.349929+0000","last_fullsized":"2024-04-23T01:53:36.349929+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":720,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":102,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.628985+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:52:29.628985+0000","last_peered":"2024-04-23T01:52:29.628985+0000","last_clean":"2024-04-23T01:52:29.628985+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:52:29.628985+0000","last_undegraded":"2024-04-23T01:52:29.628985+0000","last_fullsized":"2024-04-23T01:52:29.628985+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"48'711","reported_seq":724,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:35.661393+0000","last_change":"2024-04-23T01:52:28.529295+0000","last_active":"2024-04-23T01:53:35.661393+0000","last_peered":"2024-04-23T01:53:35.661393+0000","last_clean":"2024-04-23T01:53:35.661393+0000","last_became_active":"2024-04-23T01:52:28.523577+0000","last_became_peered":"2024-04-23T01:52:28.523577+0000","last_unstale":"2024-04-23T01:53:35.661393+0000","last_undegraded":"2024-04-23T01:53:35.661393+0000","last_fullsized":"2024-04-23T01:53:35.661393+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5832704,"num_objects":89,"num_object_clones":0,"num_object_copies":267,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":712,"num_write_kb":5696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.368290+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:52:29.368290+0000","last_peered":"2024-04-23T01:52:29.368290+0000","last_clean":"2024-04-23T01:52:29.368290+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:52:29.368290+0000","last_undegraded":"2024-04-23T01:52:29.368290+0000","last_fullsized":"2024-04-23T01:52:29.368290+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"48'517","reported_seq":530,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:36.625386+0000","last_change":"2024-04-23T01:52:28.456738+0000","last_active":"2024-04-23T01:53:36.625386+0000","last_peered":"2024-04-23T01:53:36.625386+0000","last_clean":"2024-04-23T01:53:36.625386+0000","last_became_active":"2024-04-23T01:52:28.432791+0000","last_became_peered":"2024-04-23T01:52:28.432791+0000","last_unstale":"2024-04-23T01:53:36.625386+0000","last_undegraded":"2024-04-23T01:53:36.625386+0000","last_fullsized":"2024-04-23T01:53:36.625386+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4243456,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":518,"num_write_kb":4144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":102,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.623888+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:52:29.623888+0000","last_peered":"2024-04-23T01:52:29.623888+0000","last_clean":"2024-04-23T01:52:29.623888+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:52:29.623888+0000","last_undegraded":"2024-04-23T01:52:29.623888+0000","last_fullsized":"2024-04-23T01:52:29.623888+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"48'719","reported_seq":732,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:35.496593+0000","last_change":"2024-04-23T01:52:28.427753+0000","last_active":"2024-04-23T01:53:35.496593+0000","last_peered":"2024-04-23T01:53:35.496593+0000","last_clean":"2024-04-23T01:53:35.496593+0000","last_became_active":"2024-04-23T01:52:28.423538+0000","last_became_peered":"2024-04-23T01:52:28.423538+0000","last_unstale":"2024-04-23T01:53:35.496593+0000","last_undegraded":"2024-04-23T01:53:35.496593+0000","last_fullsized":"2024-04-23T01:53:35.496593+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":720,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":108,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.278611+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:52:29.278611+0000","last_peered":"2024-04-23T01:52:29.278611+0000","last_clean":"2024-04-23T01:52:29.278611+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:52:29.278611+0000","last_undegraded":"2024-04-23T01:52:29.278611+0000","last_fullsized":"2024-04-23T01:52:29.278611+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"48'599","reported_seq":612,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:35.561310+0000","last_change":"2024-04-23T01:52:28.503432+0000","last_active":"2024-04-23T01:53:35.561310+0000","last_peered":"2024-04-23T01:53:35.561310+0000","last_clean":"2024-04-23T01:53:35.561310+0000","last_became_active":"2024-04-23T01:52:28.498953+0000","last_became_peered":"2024-04-23T01:52:28.498953+0000","last_unstale":"2024-04-23T01:53:35.561310+0000","last_undegraded":"2024-04-23T01:53:35.561310+0000","last_fullsized":"2024-04-23T01:53:35.561310+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":104,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.389394+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:52:29.389394+0000","last_peered":"2024-04-23T01:52:29.389394+0000","last_clean":"2024-04-23T01:52:29.389394+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:52:29.389394+0000","last_undegraded":"2024-04-23T01:52:29.389394+0000","last_fullsized":"2024-04-23T01:52:29.389394+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"48'479","reported_seq":492,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:35.663678+0000","last_change":"2024-04-23T01:52:28.486992+0000","last_active":"2024-04-23T01:53:35.663678+0000","last_peered":"2024-04-23T01:53:35.663678+0000","last_clean":"2024-04-23T01:53:35.663678+0000","last_became_active":"2024-04-23T01:52:28.482639+0000","last_became_peered":"2024-04-23T01:52:28.482639+0000","last_unstale":"2024-04-23T01:53:35.663678+0000","last_undegraded":"2024-04-23T01:53:35.663678+0000","last_fullsized":"2024-04-23T01:53:35.663678+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":480,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":108,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.297789+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:52:29.297789+0000","last_peered":"2024-04-23T01:52:29.297789+0000","last_clean":"2024-04-23T01:52:29.297789+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:52:29.297789+0000","last_undegraded":"2024-04-23T01:52:29.297789+0000","last_fullsized":"2024-04-23T01:52:29.297789+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"48'566","reported_seq":579,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:36.661453+0000","last_change":"2024-04-23T01:52:28.453502+0000","last_active":"2024-04-23T01:53:36.661453+0000","last_peered":"2024-04-23T01:53:36.661453+0000","last_clean":"2024-04-23T01:53:36.661453+0000","last_became_active":"2024-04-23T01:52:28.448199+0000","last_became_peered":"2024-04-23T01:52:28.448199+0000","last_unstale":"2024-04-23T01:53:36.661453+0000","last_undegraded":"2024-04-23T01:53:36.661453+0000","last_fullsized":"2024-04-23T01:53:36.661453+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4644864,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":567,"num_write_kb":4536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.366780+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:52:29.366780+0000","last_peered":"2024-04-23T01:52:29.366780+0000","last_clean":"2024-04-23T01:52:29.366780+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:52:29.366780+0000","last_undegraded":"2024-04-23T01:52:29.366780+0000","last_fullsized":"2024-04-23T01:52:29.366780+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":78823424,"num_objects":1204,"num_object_clones":0,"num_object_copies":3612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9622,"num_write_kb":76976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118235136,"data_stored":236470272,"data_compressed":12206988,"data_compressed_allocated":118235136,"data_compressed_original":236470272,"omap_allocated":0,"internal_metadata":0},"log_size":9606,"ondisk_log_size":9606,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706173,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":121356,"kb_used_data":36228,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94250484,"statfs":{"total":96636764160,"available":96512495616,"internally_reserved":0,"allocated":37097472,"data_stored":73980924,"data_compressed":3810926,"data_compressed_allocated":36663296,"data_compressed_original":73818112,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706173,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":115924,"kb_used_data":22476,"kb_used_omap":0,"kb_used_meta":93440,"kb_avail":94255916,"statfs":{"total":96636764160,"available":96518057984,"internally_reserved":0,"allocated":23015424,"data_stored":45816828,"data_compressed":2357094,"data_compressed_allocated":22581248,"data_compressed_original":45654016,"omap_allocated":0,"internal_metadata":95682560},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706173,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":120352,"kb_used_data":31064,"kb_used_omap":0,"kb_used_meta":89280,"kb_avail":94251488,"statfs":{"total":96636764160,"available":96513523712,"internally_reserved":0,"allocated":31809536,"data_stored":62937564,"data_compressed":3241441,"data_compressed_allocated":31395840,"data_compressed_original":62791680,"omap_allocated":0,"internal_metadata":91422720},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706173,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":116148,"kb_used_data":26860,"kb_used_omap":0,"kb_used_meta":89280,"kb_avail":94255692,"statfs":{"total":96636764160,"available":96517828608,"internally_reserved":0,"allocated":27504640,"data_stored":54795260,"data_compressed":2820552,"data_compressed_allocated":27070464,"data_compressed_original":54632448,"omap_allocated":0,"internal_metadata":91422720},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":27140096,"data_stored":54280192,"data_compressed":2802020,"data_compressed_allocated":27140096,"data_compressed_original":54280192,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":31576064,"data_stored":63152128,"data_compressed":3260053,"data_compressed_allocated":31576064,"data_compressed_original":63152128,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":22659072,"data_stored":45318144,"data_compressed":2339408,"data_compressed_allocated":22659072,"data_compressed_original":45318144,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":36859904,"data_stored":73719808,"data_compressed":3805507,"data_compressed_allocated":36859904,"data_compressed_original":73719808,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:53:41.302 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:53:41.345 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 72 16 10250 10234 1.11036 1.08594 0.130765 0.112488 2024-04-23T01:53:41.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:41.419+0000 7f1b23d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:41.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:41.424+0000 7f1b23d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:42.345 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 73 16 10385 10369 1.1096 1.05469 0.10667 0.112516 2024-04-23T01:53:43.346 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 74 16 10512 10496 1.10801 0.992188 0.109617 0.112662 2024-04-23T01:53:43.687 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:53:43.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:43.808+0000 7f9ea66ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:43.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:43.814+0000 7f9ea66ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:44.346 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 75 16 10633 10617 1.10584 0.945312 0.0890008 0.112945 2024-04-23T01:53:45.346 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 76 16 10749 10733 1.10321 0.90625 0.0688853 0.113178 2024-04-23T01:53:45.354 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:53:45.354 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:53:45.355 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:53:45.355 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:53:45.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:45.486+0000 7fefc495d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:45.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:45.496+0000 7fefc495d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:45.519 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:53:45.519 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:53:45.519 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:53:45.519 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:53:45.519 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:53:45.519 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:53:45.520 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:53:45.520 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:53:45.520 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:53:45.520 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:53:45.520 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:53:45.520 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:53:45.523 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:53:45.523 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:53:45.523 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:53:45.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:45.643+0000 7f17ffb9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:45.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:45.654+0000 7f17ffb9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:45.687 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:53:45.687 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:53:45.687 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:53:45.687 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:53:45.687 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:53:45.687 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:53:45.687 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:53:45.687 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:53:45.687 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:53:45.687 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:53:45.688 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:53:45.688 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:53:45.690 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:53:45.690 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:53:45.690 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:53:45.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:45.801+0000 7f31c915d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:45.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:45.810+0000 7f31c915d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:45.828 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:53:45.828 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:53:45.828 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:53:45.829 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:53:45.829 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:53:45.829 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:53:45.829 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:53:45.829 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:53:45.829 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:53:45.830 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:53:45.830 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:53:45.830 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:53:45.831 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:53:45.831 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:53:45.831 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:53:45.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:45.943+0000 7f2ffe06d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:45.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:45.953+0000 7f2ffe06d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:46.021 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:53:46.063 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:53:46.063 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:53:46.064 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:53:46.064 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:53:46.064 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:53:46.064 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:53:46.064 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:53:46.064 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:53:46.064 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:53:46.064 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:53:46.064 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:53:46.064 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:53:46.066 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:53:46.067 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:53:46.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:46.140+0000 7fedb175d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:46.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:46.147+0000 7fedb175d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:46.346 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 77 16 10883 10867 1.10248 1.04688 0.11561 0.113246 2024-04-23T01:53:47.346 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 78 16 11011 10995 1.10117 1 0.11656 0.113419 2024-04-23T01:53:48.346 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 79 16 11141 11125 1.10008 1.01562 0.0760664 0.113504 2024-04-23T01:53:48.454 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:53:48.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:48.568+0000 7f82deead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:48.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:48.574+0000 7f82deead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:49.346 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:53:49.346386+0000 min lat: 0.0154323 max lat: 0.370563 avg lat: 0.113644 2024-04-23T01:53:49.346 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:53:49.346 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 80 16 11268 11252 1.09873 0.992188 0.113236 0.113644 2024-04-23T01:53:50.346 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 81 16 11400 11384 1.0979 1.03125 0.1658 0.113734 2024-04-23T01:53:50.744 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:53:50.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:50.884+0000 7f1044e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:50.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:50.892+0000 7f1044e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:51.067 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:53:51.067 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:53:51.067 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:53:51.068 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:53:51.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:51.185+0000 7fae1f0ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:51.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:51.196+0000 7fae1f0ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:51.246 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:53:51.246 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:53:51.246 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:53:51.246 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:53:51.246 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:53:51.246 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:53:51.246 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:53:51.247 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:53:51.247 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:53:51.247 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:53:51.247 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:53:51.247 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:53:51.250 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:53:51.250 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:53:51.250 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:53:51.346 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 82 16 11537 11521 1.09756 1.07031 0.0831079 0.113792 2024-04-23T01:53:51.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:51.363+0000 7f446b55d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:51.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:51.374+0000 7f446b55d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:51.416 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:53:51.416 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:53:51.416 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:53:51.416 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:53:51.416 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:53:51.416 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:53:51.417 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:53:51.417 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:53:51.417 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:53:51.417 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:53:51.417 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:53:51.417 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:53:51.419 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:53:51.419 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:53:51.420 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:53:51.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:51.532+0000 7f2e7ee2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:51.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:51.543+0000 7f2e7ee2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:51.559 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:53:51.559 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:53:51.559 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:53:51.559 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:53:51.559 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:53:51.559 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:53:51.559 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:53:51.560 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:53:51.560 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:53:51.560 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:53:51.560 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:53:51.560 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:53:51.562 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:53:51.562 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:53:51.562 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:53:51.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:51.698+0000 7f0f420ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:51.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:51.708+0000 7f0f420ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:51.757 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:53:51.757 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:53:51.757 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:53:51.758 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:53:51.758 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:53:51.758 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:53:51.758 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:53:51.758 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:53:51.758 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:53:51.758 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:53:51.758 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:53:51.758 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:53:51.760 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:53:51.760 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:53:52.346 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 83 16 11668 11652 1.09667 1.02344 0.0237895 0.113828 2024-04-23T01:53:53.077 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:53:53.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:53.207+0000 7fb42b99d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:53.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:53.214+0000 7fb42b99d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:53.346 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 84 16 11809 11793 1.09672 1.10156 0.0538496 0.11388 2024-04-23T01:53:54.347 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 85 16 11937 11921 1.09558 1 0.0367737 0.11399 2024-04-23T01:53:55.347 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 86 16 12057 12041 1.09374 0.9375 0.146216 0.114176 2024-04-23T01:53:55.373 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:53:55.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:55.486+0000 7f0142a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:55.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:55.492+0000 7f0142a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:56.347 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 87 16 12184 12168 1.09258 0.992188 0.116809 0.114306 2024-04-23T01:53:56.761 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:53:56.761 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:53:56.761 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:53:56.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:56.870+0000 7feb51b9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:56.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:56.878+0000 7feb51b9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:57.137 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:53:57.138 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:53:57.152 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-04-23T01:53:55.650127+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":95420960,"num_objects":1450,"num_object_clones":0,"num_object_copies":4350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":11713,"num_write_kb":94502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11657,"ondisk_log_size":11657,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":488840,"kb_used_data":140008,"kb_used_omap":0,"kb_used_meta":348800,"kb_avail":376998520,"statfs":{"total":386547056640,"available":386046484480,"internally_reserved":0,"allocated":143368192,"data_stored":285412816,"data_compressed":14702445,"data_compressed_allocated":141651968,"data_compressed_original":284778496,"omap_allocated":0,"internal_metadata":357171200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10567680,"num_objects":161,"num_object_clones":0,"num_object_copies":483,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1290,"num_write_kb":10320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002261"},"pg_stats":[{"pgid":"7.f","version":"48'735","reported_seq":748,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:51.296597+0000","last_change":"2024-04-23T01:52:28.525229+0000","last_active":"2024-04-23T01:53:51.296597+0000","last_peered":"2024-04-23T01:53:51.296597+0000","last_clean":"2024-04-23T01:53:51.296597+0000","last_became_active":"2024-04-23T01:52:28.518971+0000","last_became_peered":"2024-04-23T01:52:28.518971+0000","last_unstale":"2024-04-23T01:53:51.296597+0000","last_undegraded":"2024-04-23T01:53:51.296597+0000","last_fullsized":"2024-04-23T01:53:51.296597+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6029312,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":736,"num_write_kb":5888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"48'759","reported_seq":772,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:51.092375+0000","last_change":"2024-04-23T01:52:28.530871+0000","last_active":"2024-04-23T01:53:51.092375+0000","last_peered":"2024-04-23T01:53:51.092375+0000","last_clean":"2024-04-23T01:53:51.092375+0000","last_became_active":"2024-04-23T01:52:28.521696+0000","last_became_peered":"2024-04-23T01:52:28.521696+0000","last_unstale":"2024-04-23T01:53:51.092375+0000","last_undegraded":"2024-04-23T01:53:51.092375+0000","last_fullsized":"2024-04-23T01:53:51.092375+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6225920,"num_objects":95,"num_object_clones":0,"num_object_copies":285,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":760,"num_write_kb":6080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"48'671","reported_seq":684,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:51.408550+0000","last_change":"2024-04-23T01:52:28.407738+0000","last_active":"2024-04-23T01:53:51.408550+0000","last_peered":"2024-04-23T01:53:51.408550+0000","last_clean":"2024-04-23T01:53:51.408550+0000","last_became_active":"2024-04-23T01:52:28.400739+0000","last_became_peered":"2024-04-23T01:52:28.400739+0000","last_unstale":"2024-04-23T01:53:51.408550+0000","last_undegraded":"2024-04-23T01:53:51.408550+0000","last_fullsized":"2024-04-23T01:53:51.408550+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505024,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":672,"num_write_kb":5376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"48'679","reported_seq":692,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:50.585604+0000","last_change":"2024-04-23T01:52:28.435954+0000","last_active":"2024-04-23T01:53:50.585604+0000","last_peered":"2024-04-23T01:53:50.585604+0000","last_clean":"2024-04-23T01:53:50.585604+0000","last_became_active":"2024-04-23T01:52:28.430557+0000","last_became_peered":"2024-04-23T01:52:28.430557+0000","last_unstale":"2024-04-23T01:53:50.585604+0000","last_undegraded":"2024-04-23T01:53:50.585604+0000","last_fullsized":"2024-04-23T01:53:50.585604+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5570560,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":680,"num_write_kb":5440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"48'615","reported_seq":628,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:50.383667+0000","last_change":"2024-04-23T01:52:28.452034+0000","last_active":"2024-04-23T01:53:50.383667+0000","last_peered":"2024-04-23T01:53:50.383667+0000","last_clean":"2024-04-23T01:53:50.383667+0000","last_became_active":"2024-04-23T01:52:28.430826+0000","last_became_peered":"2024-04-23T01:52:28.430826+0000","last_unstale":"2024-04-23T01:53:50.383667+0000","last_undegraded":"2024-04-23T01:53:50.383667+0000","last_fullsized":"2024-04-23T01:53:50.383667+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5046272,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":616,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"48'903","reported_seq":916,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:50.671005+0000","last_change":"2024-04-23T01:52:28.484337+0000","last_active":"2024-04-23T01:53:50.671005+0000","last_peered":"2024-04-23T01:53:50.671005+0000","last_clean":"2024-04-23T01:53:50.671005+0000","last_became_active":"2024-04-23T01:52:28.476566+0000","last_became_peered":"2024-04-23T01:52:28.476566+0000","last_unstale":"2024-04-23T01:53:50.671005+0000","last_undegraded":"2024-04-23T01:53:50.671005+0000","last_fullsized":"2024-04-23T01:53:50.671005+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":903,"log_dups_size":0,"ondisk_log_size":903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7405568,"num_objects":113,"num_object_clones":0,"num_object_copies":339,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":904,"num_write_kb":7232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"48'618","reported_seq":631,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:51.700461+0000","last_change":"2024-04-23T01:52:28.418531+0000","last_active":"2024-04-23T01:53:51.700461+0000","last_peered":"2024-04-23T01:53:51.700461+0000","last_clean":"2024-04-23T01:53:51.700461+0000","last_became_active":"2024-04-23T01:52:28.412725+0000","last_became_peered":"2024-04-23T01:52:28.412725+0000","last_unstale":"2024-04-23T01:53:51.700461+0000","last_undegraded":"2024-04-23T01:53:51.700461+0000","last_fullsized":"2024-04-23T01:53:51.700461+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5070848,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":619,"num_write_kb":4952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"48'719","reported_seq":732,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:51.347853+0000","last_change":"2024-04-23T01:52:28.498385+0000","last_active":"2024-04-23T01:53:51.347853+0000","last_peered":"2024-04-23T01:53:51.347853+0000","last_clean":"2024-04-23T01:53:51.347853+0000","last_became_active":"2024-04-23T01:52:28.487852+0000","last_became_peered":"2024-04-23T01:52:28.487852+0000","last_unstale":"2024-04-23T01:53:51.347853+0000","last_undegraded":"2024-04-23T01:53:51.347853+0000","last_fullsized":"2024-04-23T01:53:51.347853+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":720,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"48'735","reported_seq":748,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:51.641342+0000","last_change":"2024-04-23T01:52:28.536598+0000","last_active":"2024-04-23T01:53:51.641342+0000","last_peered":"2024-04-23T01:53:51.641342+0000","last_clean":"2024-04-23T01:53:51.641342+0000","last_became_active":"2024-04-23T01:52:28.530283+0000","last_became_peered":"2024-04-23T01:52:28.530283+0000","last_unstale":"2024-04-23T01:53:51.641342+0000","last_undegraded":"2024-04-23T01:53:51.641342+0000","last_fullsized":"2024-04-23T01:53:51.641342+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6029312,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":736,"num_write_kb":5888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":237,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.359805+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:52:29.359805+0000","last_peered":"2024-04-23T01:52:29.359805+0000","last_clean":"2024-04-23T01:52:29.359805+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:52:29.359805+0000","last_undegraded":"2024-04-23T01:52:29.359805+0000","last_fullsized":"2024-04-23T01:52:29.359805+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.374210+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:52:29.374210+0000","last_peered":"2024-04-23T01:52:29.374210+0000","last_clean":"2024-04-23T01:52:29.374210+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:52:29.374210+0000","last_undegraded":"2024-04-23T01:52:29.374210+0000","last_fullsized":"2024-04-23T01:52:29.374210+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"48'823","reported_seq":836,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:51.621496+0000","last_change":"2024-04-23T01:52:28.481837+0000","last_active":"2024-04-23T01:53:51.621496+0000","last_peered":"2024-04-23T01:53:51.621496+0000","last_clean":"2024-04-23T01:53:51.621496+0000","last_became_active":"2024-04-23T01:52:28.472032+0000","last_became_peered":"2024-04-23T01:52:28.472032+0000","last_unstale":"2024-04-23T01:53:51.621496+0000","last_undegraded":"2024-04-23T01:53:51.621496+0000","last_fullsized":"2024-04-23T01:53:51.621496+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":823,"log_dups_size":0,"ondisk_log_size":823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6750208,"num_objects":103,"num_object_clones":0,"num_object_copies":309,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":824,"num_write_kb":6592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":102,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.628985+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:52:29.628985+0000","last_peered":"2024-04-23T01:52:29.628985+0000","last_clean":"2024-04-23T01:52:29.628985+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:52:29.628985+0000","last_undegraded":"2024-04-23T01:52:29.628985+0000","last_fullsized":"2024-04-23T01:52:29.628985+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"48'783","reported_seq":796,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:48.759436+0000","last_change":"2024-04-23T01:52:28.529295+0000","last_active":"2024-04-23T01:53:48.759436+0000","last_peered":"2024-04-23T01:53:48.759436+0000","last_clean":"2024-04-23T01:53:48.759436+0000","last_became_active":"2024-04-23T01:52:28.523577+0000","last_became_peered":"2024-04-23T01:52:28.523577+0000","last_unstale":"2024-04-23T01:53:48.759436+0000","last_undegraded":"2024-04-23T01:53:48.759436+0000","last_fullsized":"2024-04-23T01:53:48.759436+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6422528,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.368290+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:52:29.368290+0000","last_peered":"2024-04-23T01:52:29.368290+0000","last_clean":"2024-04-23T01:52:29.368290+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:52:29.368290+0000","last_undegraded":"2024-04-23T01:52:29.368290+0000","last_fullsized":"2024-04-23T01:52:29.368290+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"48'615","reported_seq":628,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:51.510601+0000","last_change":"2024-04-23T01:52:28.456738+0000","last_active":"2024-04-23T01:53:51.510601+0000","last_peered":"2024-04-23T01:53:51.510601+0000","last_clean":"2024-04-23T01:53:51.510601+0000","last_became_active":"2024-04-23T01:52:28.432791+0000","last_became_peered":"2024-04-23T01:52:28.432791+0000","last_unstale":"2024-04-23T01:53:51.510601+0000","last_undegraded":"2024-04-23T01:53:51.510601+0000","last_fullsized":"2024-04-23T01:53:51.510601+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5046272,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":616,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":102,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.623888+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:52:29.623888+0000","last_peered":"2024-04-23T01:52:29.623888+0000","last_clean":"2024-04-23T01:52:29.623888+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:52:29.623888+0000","last_undegraded":"2024-04-23T01:52:29.623888+0000","last_fullsized":"2024-04-23T01:52:29.623888+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"48'871","reported_seq":884,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:50.983405+0000","last_change":"2024-04-23T01:52:28.427753+0000","last_active":"2024-04-23T01:53:50.983405+0000","last_peered":"2024-04-23T01:53:50.983405+0000","last_clean":"2024-04-23T01:53:50.983405+0000","last_became_active":"2024-04-23T01:52:28.423538+0000","last_became_peered":"2024-04-23T01:52:28.423538+0000","last_unstale":"2024-04-23T01:53:50.983405+0000","last_undegraded":"2024-04-23T01:53:50.983405+0000","last_fullsized":"2024-04-23T01:53:50.983405+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7143424,"num_objects":109,"num_object_clones":0,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":872,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":108,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.278611+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:52:29.278611+0000","last_peered":"2024-04-23T01:52:29.278611+0000","last_clean":"2024-04-23T01:52:29.278611+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:52:29.278611+0000","last_undegraded":"2024-04-23T01:52:29.278611+0000","last_fullsized":"2024-04-23T01:52:29.278611+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"48'702","reported_seq":715,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:51.711173+0000","last_change":"2024-04-23T01:52:28.503432+0000","last_active":"2024-04-23T01:53:51.711173+0000","last_peered":"2024-04-23T01:53:51.711173+0000","last_clean":"2024-04-23T01:53:51.711173+0000","last_became_active":"2024-04-23T01:52:28.498953+0000","last_became_peered":"2024-04-23T01:52:28.498953+0000","last_unstale":"2024-04-23T01:53:51.711173+0000","last_undegraded":"2024-04-23T01:53:51.711173+0000","last_fullsized":"2024-04-23T01:53:51.711173+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":702,"log_dups_size":0,"ondisk_log_size":702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5758976,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":703,"num_write_kb":5624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":104,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.389394+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:52:29.389394+0000","last_peered":"2024-04-23T01:52:29.389394+0000","last_clean":"2024-04-23T01:52:29.389394+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:52:29.389394+0000","last_undegraded":"2024-04-23T01:52:29.389394+0000","last_fullsized":"2024-04-23T01:52:29.389394+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"48'623","reported_seq":636,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:51.430747+0000","last_change":"2024-04-23T01:52:28.486992+0000","last_active":"2024-04-23T01:53:51.430747+0000","last_peered":"2024-04-23T01:53:51.430747+0000","last_clean":"2024-04-23T01:53:51.430747+0000","last_became_active":"2024-04-23T01:52:28.482639+0000","last_became_peered":"2024-04-23T01:52:28.482639+0000","last_unstale":"2024-04-23T01:53:51.430747+0000","last_undegraded":"2024-04-23T01:53:51.430747+0000","last_fullsized":"2024-04-23T01:53:51.430747+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5111808,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":624,"num_write_kb":4992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":108,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.297789+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:52:29.297789+0000","last_peered":"2024-04-23T01:52:29.297789+0000","last_clean":"2024-04-23T01:52:29.297789+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:52:29.297789+0000","last_undegraded":"2024-04-23T01:52:29.297789+0000","last_fullsized":"2024-04-23T01:52:29.297789+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"48'709","reported_seq":722,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:51.659571+0000","last_change":"2024-04-23T01:52:28.453502+0000","last_active":"2024-04-23T01:53:51.659571+0000","last_peered":"2024-04-23T01:53:51.659571+0000","last_clean":"2024-04-23T01:53:51.659571+0000","last_became_active":"2024-04-23T01:52:28.448199+0000","last_became_peered":"2024-04-23T01:52:28.448199+0000","last_unstale":"2024-04-23T01:53:51.659571+0000","last_undegraded":"2024-04-23T01:53:51.659571+0000","last_fullsized":"2024-04-23T01:53:51.659571+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5816320,"num_objects":89,"num_object_clones":0,"num_object_copies":267,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":710,"num_write_kb":5680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.366780+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:52:29.366780+0000","last_peered":"2024-04-23T01:52:29.366780+0000","last_clean":"2024-04-23T01:52:29.366780+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:52:29.366780+0000","last_undegraded":"2024-04-23T01:52:29.366780+0000","last_fullsized":"2024-04-23T01:52:29.366780+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":94830592,"num_objects":1448,"num_object_clones":0,"num_object_copies":4344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11576,"num_write_kb":92608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":142184448,"data_stored":284368896,"data_compressed":14680266,"data_compressed_allocated":142184448,"data_compressed_original":284368896,"omap_allocated":0,"internal_metadata":0},"log_size":11560,"ondisk_log_size":11560,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":136916,"kb_used_data":43468,"kb_used_omap":0,"kb_used_meta":93440,"kb_avail":94234924,"statfs":{"total":96636764160,"available":96496562176,"internally_reserved":0,"allocated":44511232,"data_stored":88808444,"data_compressed":4576555,"data_compressed_allocated":44077056,"data_compressed_original":88645632,"omap_allocated":0,"internal_metadata":95682560},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":107576,"kb_used_data":26608,"kb_used_omap":0,"kb_used_meta":80960,"kb_avail":94264264,"statfs":{"total":96636764160,"available":96526606336,"internally_reserved":0,"allocated":27246592,"data_stored":54279164,"data_compressed":2794053,"data_compressed_allocated":26812416,"data_compressed_original":54116352,"omap_allocated":0,"internal_metadata":82903040},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":118224,"kb_used_data":37256,"kb_used_omap":0,"kb_used_meta":80960,"kb_avail":94253616,"statfs":{"total":96636764160,"available":96515702784,"internally_reserved":0,"allocated":38150144,"data_stored":75618780,"data_compressed":3896244,"data_compressed_allocated":37736448,"data_compressed_original":75472896,"omap_allocated":0,"internal_metadata":82903040},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":126124,"kb_used_data":32676,"kb_used_omap":0,"kb_used_meta":93440,"kb_avail":94245716,"statfs":{"total":96636764160,"available":96507613184,"internally_reserved":0,"allocated":33460224,"data_stored":66706428,"data_compressed":3435593,"data_compressed_allocated":33026048,"data_compressed_original":66543616,"omap_allocated":0,"internal_metadata":95682560},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":33071104,"data_stored":66142208,"data_compressed":3414523,"data_compressed_allocated":33071104,"data_compressed_original":66142208,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":37904384,"data_stored":75808768,"data_compressed":3913587,"data_compressed_allocated":37904384,"data_compressed_original":75808768,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":26918912,"data_stored":53837824,"data_compressed":2779328,"data_compressed_allocated":26918912,"data_compressed_original":53837824,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":44290048,"data_stored":88580096,"data_compressed":4572828,"data_compressed_allocated":44290048,"data_compressed_original":88580096,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:53:57.347 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 88 16 12303 12287 1.09072 0.929688 0.0661016 0.114486 2024-04-23T01:53:57.816 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:53:57.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:57.940+0000 7f5e7a39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:57.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:53:57.946+0000 7f5e7a39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:53:58.347 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 89 16 12404 12388 1.08733 0.789062 0.208794 0.114812 2024-04-23T01:53:59.347 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 90 16 12519 12503 1.08523 0.898438 0.190635 0.115065 2024-04-23T01:54:00.110 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:54:00.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:00.213+0000 7f95b819d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:00.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:00.219+0000 7f95b819d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:00.347 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Total time run: 90.124 2024-04-23T01:54:00.348 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Total writes made: 12519 2024-04-23T01:54:00.348 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Write size: 8192 2024-04-23T01:54:00.348 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Object size: 65536 2024-04-23T01:54:00.348 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Bandwidth (MB/sec): 1.08522 2024-04-23T01:54:00.348 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Stddev Bandwidth: 0.0935979 2024-04-23T01:54:00.348 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Max bandwidth (MB/sec): 1.34375 2024-04-23T01:54:00.348 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Min bandwidth (MB/sec): 0.789062 2024-04-23T01:54:00.348 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Average IOPS: 138 2024-04-23T01:54:00.348 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Stddev IOPS: 11.9805 2024-04-23T01:54:00.348 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Max IOPS: 172 2024-04-23T01:54:00.348 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Min IOPS: 101 2024-04-23T01:54:00.348 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Average Latency(s): 0.115097 2024-04-23T01:54:00.348 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Stddev Latency(s): 0.0575583 2024-04-23T01:54:00.348 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Max latency(s): 0.374676 2024-04-23T01:54:00.349 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Min latency(s): 0.0154323 2024-04-23T01:54:00.369 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Cleaning up (deleting benchmark objects) 2024-04-23T01:54:02.155 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:54:02.155 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:54:02.155 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:54:02.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:02.271+0000 7f3502cad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:02.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:02.283+0000 7f3502cad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:02.316 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:54:02.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:02.472+0000 7efd84e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:02.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:02.480+0000 7efd84e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:02.534 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:54:02.534 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:54:02.548 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":309,"stamp":"2024-04-23T01:54:01.651299+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":100663840,"num_objects":1530,"num_object_clones":0,"num_object_copies":4590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":12353,"num_write_kb":99622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12297,"ondisk_log_size":12297,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":488320,"kb_used_data":147808,"kb_used_omap":0,"kb_used_meta":340480,"kb_avail":376999040,"statfs":{"total":386547056640,"available":386047016960,"internally_reserved":0,"allocated":151355392,"data_stored":301387216,"data_compressed":15527295,"data_compressed_allocated":149639168,"data_compressed_original":300752896,"omap_allocated":0,"internal_metadata":348651520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10625024,"num_objects":163,"num_object_clones":0,"num_object_copies":489,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1297,"num_write_kb":10376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002290"},"pg_stats":[{"pgid":"7.f","version":"48'783","reported_seq":796,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:56.421349+0000","last_change":"2024-04-23T01:52:28.525229+0000","last_active":"2024-04-23T01:53:56.421349+0000","last_peered":"2024-04-23T01:53:56.421349+0000","last_clean":"2024-04-23T01:53:56.421349+0000","last_became_active":"2024-04-23T01:52:28.518971+0000","last_became_peered":"2024-04-23T01:52:28.518971+0000","last_unstale":"2024-04-23T01:53:56.421349+0000","last_undegraded":"2024-04-23T01:53:56.421349+0000","last_fullsized":"2024-04-23T01:53:56.421349+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6422528,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"48'823","reported_seq":836,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:56.092707+0000","last_change":"2024-04-23T01:52:28.530871+0000","last_active":"2024-04-23T01:53:56.092707+0000","last_peered":"2024-04-23T01:53:56.092707+0000","last_clean":"2024-04-23T01:53:56.092707+0000","last_became_active":"2024-04-23T01:52:28.521696+0000","last_became_peered":"2024-04-23T01:52:28.521696+0000","last_unstale":"2024-04-23T01:53:56.092707+0000","last_undegraded":"2024-04-23T01:53:56.092707+0000","last_fullsized":"2024-04-23T01:53:56.092707+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":823,"log_dups_size":0,"ondisk_log_size":823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6750208,"num_objects":103,"num_object_clones":0,"num_object_copies":309,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":824,"num_write_kb":6592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"48'727","reported_seq":740,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:55.292537+0000","last_change":"2024-04-23T01:52:28.407738+0000","last_active":"2024-04-23T01:53:55.292537+0000","last_peered":"2024-04-23T01:53:55.292537+0000","last_clean":"2024-04-23T01:53:55.292537+0000","last_became_active":"2024-04-23T01:52:28.400739+0000","last_became_peered":"2024-04-23T01:52:28.400739+0000","last_unstale":"2024-04-23T01:53:55.292537+0000","last_undegraded":"2024-04-23T01:53:55.292537+0000","last_fullsized":"2024-04-23T01:53:55.292537+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":728,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"48'719","reported_seq":732,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:56.706749+0000","last_change":"2024-04-23T01:52:28.435954+0000","last_active":"2024-04-23T01:53:56.706749+0000","last_peered":"2024-04-23T01:53:56.706749+0000","last_clean":"2024-04-23T01:53:56.706749+0000","last_became_active":"2024-04-23T01:52:28.430557+0000","last_became_peered":"2024-04-23T01:52:28.430557+0000","last_unstale":"2024-04-23T01:53:56.706749+0000","last_undegraded":"2024-04-23T01:53:56.706749+0000","last_fullsized":"2024-04-23T01:53:56.706749+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":720,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"48'652","reported_seq":665,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:56.674721+0000","last_change":"2024-04-23T01:52:28.452034+0000","last_active":"2024-04-23T01:53:56.674721+0000","last_peered":"2024-04-23T01:53:56.674721+0000","last_clean":"2024-04-23T01:53:56.674721+0000","last_became_active":"2024-04-23T01:52:28.430826+0000","last_became_peered":"2024-04-23T01:52:28.430826+0000","last_unstale":"2024-04-23T01:53:56.674721+0000","last_undegraded":"2024-04-23T01:53:56.674721+0000","last_fullsized":"2024-04-23T01:53:56.674721+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5349376,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":653,"num_write_kb":5224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"48'951","reported_seq":964,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:55.815413+0000","last_change":"2024-04-23T01:52:28.484337+0000","last_active":"2024-04-23T01:53:55.815413+0000","last_peered":"2024-04-23T01:53:55.815413+0000","last_clean":"2024-04-23T01:53:55.815413+0000","last_became_active":"2024-04-23T01:52:28.476566+0000","last_became_peered":"2024-04-23T01:52:28.476566+0000","last_unstale":"2024-04-23T01:53:55.815413+0000","last_undegraded":"2024-04-23T01:53:55.815413+0000","last_fullsized":"2024-04-23T01:53:55.815413+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":951,"log_dups_size":0,"ondisk_log_size":951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7798784,"num_objects":119,"num_object_clones":0,"num_object_copies":357,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":952,"num_write_kb":7616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"48'639","reported_seq":652,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:53.502907+0000","last_change":"2024-04-23T01:52:28.418531+0000","last_active":"2024-04-23T01:53:53.502907+0000","last_peered":"2024-04-23T01:53:53.502907+0000","last_clean":"2024-04-23T01:53:53.502907+0000","last_became_active":"2024-04-23T01:52:28.412725+0000","last_became_peered":"2024-04-23T01:52:28.412725+0000","last_unstale":"2024-04-23T01:53:53.502907+0000","last_undegraded":"2024-04-23T01:53:53.502907+0000","last_fullsized":"2024-04-23T01:53:53.502907+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":640,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"48'751","reported_seq":764,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:55.578950+0000","last_change":"2024-04-23T01:52:28.498385+0000","last_active":"2024-04-23T01:53:55.578950+0000","last_peered":"2024-04-23T01:53:55.578950+0000","last_clean":"2024-04-23T01:53:55.578950+0000","last_became_active":"2024-04-23T01:52:28.487852+0000","last_became_peered":"2024-04-23T01:52:28.487852+0000","last_unstale":"2024-04-23T01:53:55.578950+0000","last_undegraded":"2024-04-23T01:53:55.578950+0000","last_fullsized":"2024-04-23T01:53:55.578950+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"48'783","reported_seq":796,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:56.533795+0000","last_change":"2024-04-23T01:52:28.536598+0000","last_active":"2024-04-23T01:53:56.533795+0000","last_peered":"2024-04-23T01:53:56.533795+0000","last_clean":"2024-04-23T01:53:56.533795+0000","last_became_active":"2024-04-23T01:52:28.530283+0000","last_became_peered":"2024-04-23T01:52:28.530283+0000","last_unstale":"2024-04-23T01:53:56.533795+0000","last_undegraded":"2024-04-23T01:53:56.533795+0000","last_fullsized":"2024-04-23T01:53:56.533795+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6422528,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":237,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.359805+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:52:29.359805+0000","last_peered":"2024-04-23T01:52:29.359805+0000","last_clean":"2024-04-23T01:52:29.359805+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:52:29.359805+0000","last_undegraded":"2024-04-23T01:52:29.359805+0000","last_fullsized":"2024-04-23T01:52:29.359805+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.374210+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:52:29.374210+0000","last_peered":"2024-04-23T01:52:29.374210+0000","last_clean":"2024-04-23T01:52:29.374210+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:52:29.374210+0000","last_undegraded":"2024-04-23T01:52:29.374210+0000","last_fullsized":"2024-04-23T01:52:29.374210+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"48'855","reported_seq":868,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:56.005873+0000","last_change":"2024-04-23T01:52:28.481837+0000","last_active":"2024-04-23T01:53:56.005873+0000","last_peered":"2024-04-23T01:53:56.005873+0000","last_clean":"2024-04-23T01:53:56.005873+0000","last_became_active":"2024-04-23T01:52:28.472032+0000","last_became_peered":"2024-04-23T01:52:28.472032+0000","last_unstale":"2024-04-23T01:53:56.005873+0000","last_undegraded":"2024-04-23T01:53:56.005873+0000","last_fullsized":"2024-04-23T01:53:56.005873+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7012352,"num_objects":107,"num_object_clones":0,"num_object_copies":321,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":856,"num_write_kb":6848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":102,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.628985+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:52:29.628985+0000","last_peered":"2024-04-23T01:52:29.628985+0000","last_clean":"2024-04-23T01:52:29.628985+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:52:29.628985+0000","last_undegraded":"2024-04-23T01:52:29.628985+0000","last_fullsized":"2024-04-23T01:52:29.628985+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"48'815","reported_seq":828,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:56.174210+0000","last_change":"2024-04-23T01:52:28.529295+0000","last_active":"2024-04-23T01:53:56.174210+0000","last_peered":"2024-04-23T01:53:56.174210+0000","last_clean":"2024-04-23T01:53:56.174210+0000","last_became_active":"2024-04-23T01:52:28.523577+0000","last_became_peered":"2024-04-23T01:52:28.523577+0000","last_unstale":"2024-04-23T01:53:56.174210+0000","last_undegraded":"2024-04-23T01:53:56.174210+0000","last_fullsized":"2024-04-23T01:53:56.174210+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6684672,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":816,"num_write_kb":6528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.368290+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:52:29.368290+0000","last_peered":"2024-04-23T01:52:29.368290+0000","last_clean":"2024-04-23T01:52:29.368290+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:52:29.368290+0000","last_undegraded":"2024-04-23T01:52:29.368290+0000","last_fullsized":"2024-04-23T01:52:29.368290+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"48'650","reported_seq":663,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:56.640065+0000","last_change":"2024-04-23T01:52:28.456738+0000","last_active":"2024-04-23T01:53:56.640065+0000","last_peered":"2024-04-23T01:53:56.640065+0000","last_clean":"2024-04-23T01:53:56.640065+0000","last_became_active":"2024-04-23T01:52:28.432791+0000","last_became_peered":"2024-04-23T01:52:28.432791+0000","last_unstale":"2024-04-23T01:53:56.640065+0000","last_undegraded":"2024-04-23T01:53:56.640065+0000","last_fullsized":"2024-04-23T01:53:56.640065+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5332992,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":651,"num_write_kb":5208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":102,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.623888+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:52:29.623888+0000","last_peered":"2024-04-23T01:52:29.623888+0000","last_clean":"2024-04-23T01:52:29.623888+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:52:29.623888+0000","last_undegraded":"2024-04-23T01:52:29.623888+0000","last_fullsized":"2024-04-23T01:52:29.623888+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"48'943","reported_seq":956,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:56.323852+0000","last_change":"2024-04-23T01:52:28.427753+0000","last_active":"2024-04-23T01:53:56.323852+0000","last_peered":"2024-04-23T01:53:56.323852+0000","last_clean":"2024-04-23T01:53:56.323852+0000","last_became_active":"2024-04-23T01:52:28.423538+0000","last_became_peered":"2024-04-23T01:52:28.423538+0000","last_unstale":"2024-04-23T01:53:56.323852+0000","last_undegraded":"2024-04-23T01:53:56.323852+0000","last_fullsized":"2024-04-23T01:53:56.323852+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":943,"log_dups_size":0,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7733248,"num_objects":118,"num_object_clones":0,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":944,"num_write_kb":7552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":108,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.278611+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:52:29.278611+0000","last_peered":"2024-04-23T01:52:29.278611+0000","last_clean":"2024-04-23T01:52:29.278611+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:52:29.278611+0000","last_undegraded":"2024-04-23T01:52:29.278611+0000","last_fullsized":"2024-04-23T01:52:29.278611+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"48'735","reported_seq":748,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:53.307864+0000","last_change":"2024-04-23T01:52:28.503432+0000","last_active":"2024-04-23T01:53:53.307864+0000","last_peered":"2024-04-23T01:53:53.307864+0000","last_clean":"2024-04-23T01:53:53.307864+0000","last_became_active":"2024-04-23T01:52:28.498953+0000","last_became_peered":"2024-04-23T01:52:28.498953+0000","last_unstale":"2024-04-23T01:53:53.307864+0000","last_undegraded":"2024-04-23T01:53:53.307864+0000","last_fullsized":"2024-04-23T01:53:53.307864+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6029312,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":736,"num_write_kb":5888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":104,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.389394+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:52:29.389394+0000","last_peered":"2024-04-23T01:52:29.389394+0000","last_clean":"2024-04-23T01:52:29.389394+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:52:29.389394+0000","last_undegraded":"2024-04-23T01:52:29.389394+0000","last_fullsized":"2024-04-23T01:52:29.389394+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"48'647","reported_seq":660,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:55.935581+0000","last_change":"2024-04-23T01:52:28.486992+0000","last_active":"2024-04-23T01:53:55.935581+0000","last_peered":"2024-04-23T01:53:55.935581+0000","last_clean":"2024-04-23T01:53:55.935581+0000","last_became_active":"2024-04-23T01:52:28.482639+0000","last_became_peered":"2024-04-23T01:52:28.482639+0000","last_unstale":"2024-04-23T01:53:55.935581+0000","last_undegraded":"2024-04-23T01:53:55.935581+0000","last_fullsized":"2024-04-23T01:53:55.935581+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308416,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":648,"num_write_kb":5184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":108,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.297789+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:52:29.297789+0000","last_peered":"2024-04-23T01:52:29.297789+0000","last_clean":"2024-04-23T01:52:29.297789+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:52:29.297789+0000","last_undegraded":"2024-04-23T01:52:29.297789+0000","last_fullsized":"2024-04-23T01:52:29.297789+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"48'727","reported_seq":740,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:53:56.499583+0000","last_change":"2024-04-23T01:52:28.453502+0000","last_active":"2024-04-23T01:53:56.499583+0000","last_peered":"2024-04-23T01:53:56.499583+0000","last_clean":"2024-04-23T01:53:56.499583+0000","last_became_active":"2024-04-23T01:52:28.448199+0000","last_became_peered":"2024-04-23T01:52:28.448199+0000","last_unstale":"2024-04-23T01:53:56.499583+0000","last_undegraded":"2024-04-23T01:53:56.499583+0000","last_fullsized":"2024-04-23T01:53:56.499583+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":728,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.366780+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:52:29.366780+0000","last_peered":"2024-04-23T01:52:29.366780+0000","last_clean":"2024-04-23T01:52:29.366780+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:52:29.366780+0000","last_undegraded":"2024-04-23T01:52:29.366780+0000","last_fullsized":"2024-04-23T01:52:29.366780+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":100073472,"num_objects":1528,"num_object_clones":0,"num_object_copies":4584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12216,"num_write_kb":97728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":150065152,"data_stored":300130304,"data_compressed":15494118,"data_compressed_allocated":150065152,"data_compressed_original":300130304,"omap_allocated":0,"internal_metadata":0},"log_size":12200,"ondisk_log_size":12200,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706193,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":139264,"kb_used_data":45816,"kb_used_omap":0,"kb_used_meta":93440,"kb_avail":94232576,"statfs":{"total":96636764160,"available":96494157824,"internally_reserved":0,"allocated":46915584,"data_stored":93617148,"data_compressed":4824856,"data_compressed_allocated":46481408,"data_compressed_original":93454336,"omap_allocated":0,"internal_metadata":95682560},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706193,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":109080,"kb_used_data":28112,"kb_used_omap":0,"kb_used_meta":80960,"kb_avail":94262760,"statfs":{"total":96636764160,"available":96525066240,"internally_reserved":0,"allocated":28786688,"data_stored":57359356,"data_compressed":2953101,"data_compressed_allocated":28352512,"data_compressed_original":57196544,"omap_allocated":0,"internal_metadata":82903040},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706193,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":124532,"kb_used_data":39404,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94247308,"statfs":{"total":96636764160,"available":96509243392,"internally_reserved":0,"allocated":40349696,"data_stored":80017884,"data_compressed":4123395,"data_compressed_allocated":39936000,"data_compressed_original":79872000,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706193,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":115444,"kb_used_data":34476,"kb_used_omap":0,"kb_used_meta":80960,"kb_avail":94256396,"statfs":{"total":96636764160,"available":96518549504,"internally_reserved":0,"allocated":35303424,"data_stored":70392828,"data_compressed":3625943,"data_compressed_allocated":34869248,"data_compressed_original":70230016,"omap_allocated":0,"internal_metadata":82903040},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":34942976,"data_stored":69885952,"data_compressed":3607834,"data_compressed_allocated":34942976,"data_compressed_original":69885952,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40116224,"data_stored":80232448,"data_compressed":4142007,"data_compressed_allocated":40116224,"data_compressed_original":80232448,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28348416,"data_stored":56696832,"data_compressed":2926955,"data_compressed_allocated":28348416,"data_compressed_original":56696832,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":46657536,"data_stored":93315072,"data_compressed":4817322,"data_compressed_allocated":46657536,"data_compressed_original":93315072,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:54:04.900 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:54:05.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:05.024+0000 7f1ac915d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:05.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:05.031+0000 7f1ac915d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:07.551 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:54:07.552 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:54:07.552 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:54:07.577 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:54:07.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:07.707+0000 7fdea9c24640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:07.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:07.717+0000 7fdea9c24640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:07.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:07.716+0000 7f7de5c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:07.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:07.723+0000 7f7de5c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:07.963 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:54:07.963 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:54:07.979 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2024-04-23T01:54:07.652544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":38986296,"num_objects":589,"num_object_clones":0,"num_object_copies":1767,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":13636,"num_write_kb":102047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13580,"ondisk_log_size":13580,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":400900,"kb_used_data":64516,"kb_used_omap":0,"kb_used_meta":336320,"kb_avail":377086460,"statfs":{"total":386547056640,"available":386136535040,"internally_reserved":0,"allocated":66064384,"data_stored":130780696,"data_compressed":6721227,"data_compressed_allocated":64335872,"data_compressed_original":130146304,"omap_allocated":0,"internal_metadata":344391680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-56434664,"num_objects":-861,"num_object_clones":0,"num_object_copies":-2583,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1923,"num_write_kb":7545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002416"},"pg_stats":[{"pgid":"7.f","version":"48'848","reported_seq":861,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:54:06.188806+0000","last_change":"2024-04-23T01:52:28.525229+0000","last_active":"2024-04-23T01:54:06.188806+0000","last_peered":"2024-04-23T01:54:06.188806+0000","last_clean":"2024-04-23T01:54:06.188806+0000","last_became_active":"2024-04-23T01:52:28.518971+0000","last_became_peered":"2024-04-23T01:52:28.518971+0000","last_unstale":"2024-04-23T01:54:06.188806+0000","last_undegraded":"2024-04-23T01:54:06.188806+0000","last_fullsized":"2024-04-23T01:54:06.188806+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":848,"log_dups_size":0,"ondisk_log_size":848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":849,"num_write_kb":6336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"48'917","reported_seq":930,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:54:06.337025+0000","last_change":"2024-04-23T01:52:28.530871+0000","last_active":"2024-04-23T01:54:06.337025+0000","last_peered":"2024-04-23T01:54:06.337025+0000","last_clean":"2024-04-23T01:54:06.337025+0000","last_became_active":"2024-04-23T01:52:28.521696+0000","last_became_peered":"2024-04-23T01:52:28.521696+0000","last_unstale":"2024-04-23T01:54:06.337025+0000","last_undegraded":"2024-04-23T01:54:06.337025+0000","last_fullsized":"2024-04-23T01:54:06.337025+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":917,"log_dups_size":0,"ondisk_log_size":917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2949120,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":918,"num_write_kb":6848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"48'812","reported_seq":826,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:54:06.645918+0000","last_change":"2024-04-23T01:52:28.407738+0000","last_active":"2024-04-23T01:54:06.645918+0000","last_peered":"2024-04-23T01:54:06.645918+0000","last_clean":"2024-04-23T01:54:06.645918+0000","last_became_active":"2024-04-23T01:52:28.400739+0000","last_became_peered":"2024-04-23T01:52:28.400739+0000","last_unstale":"2024-04-23T01:54:06.645918+0000","last_undegraded":"2024-04-23T01:54:06.645918+0000","last_fullsized":"2024-04-23T01:54:06.645918+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":812,"log_dups_size":0,"ondisk_log_size":812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228248,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":813,"num_write_kb":6017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"48'808","reported_seq":821,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:54:06.526906+0000","last_change":"2024-04-23T01:52:28.435954+0000","last_active":"2024-04-23T01:54:06.526906+0000","last_peered":"2024-04-23T01:54:06.526906+0000","last_clean":"2024-04-23T01:54:06.526906+0000","last_became_active":"2024-04-23T01:52:28.430557+0000","last_became_peered":"2024-04-23T01:52:28.430557+0000","last_unstale":"2024-04-23T01:54:06.526906+0000","last_undegraded":"2024-04-23T01:54:06.526906+0000","last_fullsized":"2024-04-23T01:54:06.526906+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":808,"log_dups_size":0,"ondisk_log_size":808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2424832,"num_objects":37,"num_object_clones":0,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":809,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"48'707","reported_seq":720,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:54:06.553043+0000","last_change":"2024-04-23T01:52:28.452034+0000","last_active":"2024-04-23T01:54:06.553043+0000","last_peered":"2024-04-23T01:54:06.553043+0000","last_clean":"2024-04-23T01:54:06.553043+0000","last_became_active":"2024-04-23T01:52:28.430826+0000","last_became_peered":"2024-04-23T01:52:28.430826+0000","last_unstale":"2024-04-23T01:54:06.553043+0000","last_undegraded":"2024-04-23T01:54:06.553043+0000","last_fullsized":"2024-04-23T01:54:06.553043+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":708,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"48'1054","reported_seq":1067,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:54:06.497919+0000","last_change":"2024-04-23T01:52:28.484337+0000","last_active":"2024-04-23T01:54:06.497919+0000","last_peered":"2024-04-23T01:54:06.497919+0000","last_clean":"2024-04-23T01:54:06.497919+0000","last_became_active":"2024-04-23T01:52:28.476566+0000","last_became_peered":"2024-04-23T01:52:28.476566+0000","last_unstale":"2024-04-23T01:54:06.497919+0000","last_undegraded":"2024-04-23T01:54:06.497919+0000","last_fullsized":"2024-04-23T01:54:06.497919+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":1054,"log_dups_size":0,"ondisk_log_size":1054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3334144,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1055,"num_write_kb":7864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"48'714","reported_seq":727,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:54:06.408261+0000","last_change":"2024-04-23T01:52:28.418531+0000","last_active":"2024-04-23T01:54:06.408261+0000","last_peered":"2024-04-23T01:54:06.408261+0000","last_clean":"2024-04-23T01:54:06.408261+0000","last_became_active":"2024-04-23T01:52:28.412725+0000","last_became_peered":"2024-04-23T01:52:28.412725+0000","last_unstale":"2024-04-23T01:54:06.408261+0000","last_undegraded":"2024-04-23T01:54:06.408261+0000","last_fullsized":"2024-04-23T01:54:06.408261+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":715,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"48'833","reported_seq":846,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:54:06.459115+0000","last_change":"2024-04-23T01:52:28.498385+0000","last_active":"2024-04-23T01:54:06.459115+0000","last_peered":"2024-04-23T01:54:06.459115+0000","last_clean":"2024-04-23T01:54:06.459115+0000","last_became_active":"2024-04-23T01:52:28.487852+0000","last_became_peered":"2024-04-23T01:52:28.487852+0000","last_unstale":"2024-04-23T01:54:06.459115+0000","last_undegraded":"2024-04-23T01:54:06.459115+0000","last_fullsized":"2024-04-23T01:54:06.459115+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":834,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"48'867","reported_seq":880,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:54:06.681190+0000","last_change":"2024-04-23T01:52:28.536598+0000","last_active":"2024-04-23T01:54:06.681190+0000","last_peered":"2024-04-23T01:54:06.681190+0000","last_clean":"2024-04-23T01:54:06.681190+0000","last_became_active":"2024-04-23T01:52:28.530283+0000","last_became_peered":"2024-04-23T01:52:28.530283+0000","last_unstale":"2024-04-23T01:54:06.681190+0000","last_undegraded":"2024-04-23T01:54:06.681190+0000","last_fullsized":"2024-04-23T01:54:06.681190+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":867,"log_dups_size":0,"ondisk_log_size":867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":32,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":868,"num_write_kb":6400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":237,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.359805+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:52:29.359805+0000","last_peered":"2024-04-23T01:52:29.359805+0000","last_clean":"2024-04-23T01:52:29.359805+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:52:29.359805+0000","last_undegraded":"2024-04-23T01:52:29.359805+0000","last_fullsized":"2024-04-23T01:52:29.359805+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.374210+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:52:29.374210+0000","last_peered":"2024-04-23T01:52:29.374210+0000","last_clean":"2024-04-23T01:52:29.374210+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:52:29.374210+0000","last_undegraded":"2024-04-23T01:52:29.374210+0000","last_fullsized":"2024-04-23T01:52:29.374210+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"48'946","reported_seq":959,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:54:06.547597+0000","last_change":"2024-04-23T01:52:28.481837+0000","last_active":"2024-04-23T01:54:06.547597+0000","last_peered":"2024-04-23T01:54:06.547597+0000","last_clean":"2024-04-23T01:54:06.547597+0000","last_became_active":"2024-04-23T01:52:28.472032+0000","last_became_peered":"2024-04-23T01:52:28.472032+0000","last_unstale":"2024-04-23T01:54:06.547597+0000","last_undegraded":"2024-04-23T01:54:06.547597+0000","last_fullsized":"2024-04-23T01:54:06.547597+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":946,"log_dups_size":0,"ondisk_log_size":946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":947,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":102,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.628985+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:52:29.628985+0000","last_peered":"2024-04-23T01:52:29.628985+0000","last_clean":"2024-04-23T01:52:29.628985+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:52:29.628985+0000","last_undegraded":"2024-04-23T01:52:29.628985+0000","last_fullsized":"2024-04-23T01:52:29.628985+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"48'895","reported_seq":908,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:54:06.661945+0000","last_change":"2024-04-23T01:52:28.529295+0000","last_active":"2024-04-23T01:54:06.661945+0000","last_peered":"2024-04-23T01:54:06.661945+0000","last_clean":"2024-04-23T01:54:06.661945+0000","last_became_active":"2024-04-23T01:52:28.523577+0000","last_became_peered":"2024-04-23T01:52:28.523577+0000","last_unstale":"2024-04-23T01:54:06.661945+0000","last_undegraded":"2024-04-23T01:54:06.661945+0000","last_fullsized":"2024-04-23T01:54:06.661945+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":896,"num_write_kb":6592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.368290+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:52:29.368290+0000","last_peered":"2024-04-23T01:52:29.368290+0000","last_clean":"2024-04-23T01:52:29.368290+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:52:29.368290+0000","last_undegraded":"2024-04-23T01:52:29.368290+0000","last_fullsized":"2024-04-23T01:52:29.368290+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"48'710","reported_seq":723,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:54:06.488743+0000","last_change":"2024-04-23T01:52:28.456738+0000","last_active":"2024-04-23T01:54:06.488743+0000","last_peered":"2024-04-23T01:54:06.488743+0000","last_clean":"2024-04-23T01:54:06.488743+0000","last_became_active":"2024-04-23T01:52:28.432791+0000","last_became_peered":"2024-04-23T01:52:28.432791+0000","last_unstale":"2024-04-23T01:54:06.488743+0000","last_undegraded":"2024-04-23T01:54:06.488743+0000","last_fullsized":"2024-04-23T01:54:06.488743+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1769472,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":711,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":102,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.623888+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:52:29.623888+0000","last_peered":"2024-04-23T01:52:29.623888+0000","last_clean":"2024-04-23T01:52:29.623888+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:52:29.623888+0000","last_undegraded":"2024-04-23T01:52:29.623888+0000","last_fullsized":"2024-04-23T01:52:29.623888+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"48'1027","reported_seq":1040,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:54:06.636790+0000","last_change":"2024-04-23T01:52:28.427753+0000","last_active":"2024-04-23T01:54:06.636790+0000","last_peered":"2024-04-23T01:54:06.636790+0000","last_clean":"2024-04-23T01:54:06.636790+0000","last_became_active":"2024-04-23T01:52:28.423538+0000","last_became_peered":"2024-04-23T01:52:28.423538+0000","last_unstale":"2024-04-23T01:54:06.636790+0000","last_undegraded":"2024-04-23T01:54:06.636790+0000","last_fullsized":"2024-04-23T01:54:06.636790+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":1027,"log_dups_size":0,"ondisk_log_size":1027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1028,"num_write_kb":7616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":108,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.278611+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:52:29.278611+0000","last_peered":"2024-04-23T01:52:29.278611+0000","last_clean":"2024-04-23T01:52:29.278611+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:52:29.278611+0000","last_undegraded":"2024-04-23T01:52:29.278611+0000","last_fullsized":"2024-04-23T01:52:29.278611+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"48'795","reported_seq":808,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:54:06.324088+0000","last_change":"2024-04-23T01:52:28.503432+0000","last_active":"2024-04-23T01:54:06.324088+0000","last_peered":"2024-04-23T01:54:06.324088+0000","last_clean":"2024-04-23T01:54:06.324088+0000","last_became_active":"2024-04-23T01:52:28.498953+0000","last_became_peered":"2024-04-23T01:52:28.498953+0000","last_unstale":"2024-04-23T01:54:06.324088+0000","last_undegraded":"2024-04-23T01:54:06.324088+0000","last_fullsized":"2024-04-23T01:54:06.324088+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":795,"log_dups_size":0,"ondisk_log_size":795,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2686976,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":796,"num_write_kb":5952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":104,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.389394+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:52:29.389394+0000","last_peered":"2024-04-23T01:52:29.389394+0000","last_clean":"2024-04-23T01:52:29.389394+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:52:29.389394+0000","last_undegraded":"2024-04-23T01:52:29.389394+0000","last_fullsized":"2024-04-23T01:52:29.389394+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"48'746","reported_seq":759,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:54:06.657516+0000","last_change":"2024-04-23T01:52:28.486992+0000","last_active":"2024-04-23T01:54:06.657516+0000","last_peered":"2024-04-23T01:54:06.657516+0000","last_clean":"2024-04-23T01:54:06.657516+0000","last_became_active":"2024-04-23T01:52:28.482639+0000","last_became_peered":"2024-04-23T01:52:28.482639+0000","last_unstale":"2024-04-23T01:54:06.657516+0000","last_undegraded":"2024-04-23T01:54:06.657516+0000","last_fullsized":"2024-04-23T01:54:06.657516+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":746,"log_dups_size":0,"ondisk_log_size":746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":747,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":108,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.297789+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:52:29.297789+0000","last_peered":"2024-04-23T01:52:29.297789+0000","last_clean":"2024-04-23T01:52:29.297789+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:52:29.297789+0000","last_undegraded":"2024-04-23T01:52:29.297789+0000","last_fullsized":"2024-04-23T01:52:29.297789+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"48'804","reported_seq":817,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:54:06.549154+0000","last_change":"2024-04-23T01:52:28.453502+0000","last_active":"2024-04-23T01:54:06.549154+0000","last_peered":"2024-04-23T01:54:06.549154+0000","last_clean":"2024-04-23T01:54:06.549154+0000","last_became_active":"2024-04-23T01:52:28.448199+0000","last_became_peered":"2024-04-23T01:52:28.448199+0000","last_unstale":"2024-04-23T01:54:06.549154+0000","last_undegraded":"2024-04-23T01:54:06.549154+0000","last_fullsized":"2024-04-23T01:54:06.549154+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:52:27.246399+0000","last_clean_scrub_stamp":"2024-04-23T01:52:27.246399+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2686976,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":805,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-23T01:52:29.366780+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:52:29.366780+0000","last_peered":"2024-04-23T01:52:29.366780+0000","last_clean":"2024-04-23T01:52:29.366780+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:52:29.366780+0000","last_undegraded":"2024-04-23T01:52:29.366780+0000","last_fullsized":"2024-04-23T01:52:29.366780+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":38395928,"num_objects":587,"num_object_clones":0,"num_object_copies":1761,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13499,"num_write_kb":100153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":58064896,"data_stored":116105288,"data_compressed":5995176,"data_compressed_allocated":58052608,"data_compressed_original":116105216,"omap_allocated":0,"internal_metadata":0},"log_size":13483,"ondisk_log_size":13483,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706203,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":101016,"kb_used_data":20048,"kb_used_omap":0,"kb_used_meta":80960,"kb_avail":94270824,"statfs":{"total":96636764160,"available":96533323776,"internally_reserved":0,"allocated":20529152,"data_stored":40836116,"data_compressed":2100537,"data_compressed_allocated":20090880,"data_compressed_original":40673280,"omap_allocated":0,"internal_metadata":82903040},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706203,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":97752,"kb_used_data":12624,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94274088,"statfs":{"total":96636764160,"available":96536666112,"internally_reserved":0,"allocated":12926976,"data_stored":25631764,"data_compressed":1315450,"data_compressed_allocated":12488704,"data_compressed_original":25468928,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706203,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":102208,"kb_used_data":17048,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94269632,"statfs":{"total":96636764160,"available":96532103168,"internally_reserved":0,"allocated":17457152,"data_stored":34224628,"data_compressed":1759679,"data_compressed_allocated":17039360,"data_compressed_original":34078720,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706203,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":99924,"kb_used_data":14796,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94271916,"statfs":{"total":96636764160,"available":96534441984,"internally_reserved":0,"allocated":15151104,"data_stored":30088188,"data_compressed":1545561,"data_compressed_allocated":14716928,"data_compressed_original":29925376,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13627392,"data_stored":27254784,"data_compressed":1407320,"data_compressed_allocated":13627392,"data_compressed_original":27254784,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15536128,"data_stored":31064088,"data_compressed":1604015,"data_compressed_allocated":15532032,"data_compressed_original":31064064,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":11108352,"data_stored":22208536,"data_compressed":1146753,"data_compressed_allocated":11104256,"data_compressed_original":22208512,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":17793024,"data_stored":35577880,"data_compressed":1837088,"data_compressed_allocated":17788928,"data_compressed_original":35577856,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:54:10.190 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:54:10.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:10.334+0000 7ff16919d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:10.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:10.347+0000 7ff16919d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:10.601 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Removed 1565 objects 2024-04-23T01:54:10.601 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Clean up completed and total clean up time :10.2315 2024-04-23T01:54:12.657 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:54:12.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:12.765+0000 7fe0254ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:12.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:12.771+0000 7fe0254ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:12.981 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:54:12.982 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:54:12.982 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:54:12.982 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:54:13.004 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_4 2024-04-23T01:54:13.004 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_4 unique_pool_4 --yes-i-really-really-mean-it 2024-04-23T01:54:13.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:13.086+0000 7facbb19d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:13.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:13.095+0000 7facbb19d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:13.113 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:54:13.113 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:54:13.113 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:54:13.113 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:54:13.113 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:54:13.113 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:54:13.113 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:54:13.113 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:54:13.113 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:54:13.114 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:54:13.114 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:54:13.114 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:54:13.116 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:54:13.117 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:54:13.117 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:54:13.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:13.130+0000 7f638859d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:13.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:13.144+0000 7f638859d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:13.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:13.223+0000 7f5e6a617640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:13.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:13.232+0000 7f5e6a617640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:13.249 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:54:13.249 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:54:13.249 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:54:13.249 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:54:13.249 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:54:13.250 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:54:13.250 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:54:13.250 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:54:13.250 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:54:13.250 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:54:13.250 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:54:13.250 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:54:13.252 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:54:13.253 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:54:13.253 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:54:13.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:13.395+0000 7f5604aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:13.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:13.403+0000 7f5604aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:13.416 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'unique_pool_4' removed 2024-04-23T01:54:13.423 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:54:13.423 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:54:13.423 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:54:13.423 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:54:13.423 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:54:13.423 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:54:13.423 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:54:13.423 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:54:13.423 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:54:13.423 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:54:13.423 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:54:13.423 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:54:13.426 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:54:13.426 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:54:13.427 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:54:13.432 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-04-23T01:54:13.433 INFO:tasks.radosbench:Beginning radosbench... 2024-04-23T01:54:13.433 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_5 2024-04-23T01:54:13.434 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_5 16 2024-04-23T01:54:13.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:13.538+0000 7fe5c2a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:13.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:13.548+0000 7fe5c2a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:13.570 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:54:13.570 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:54:13.570 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:54:13.571 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:54:13.571 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:54:13.571 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:54:13.571 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:54:13.571 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:54:13.571 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:54:13.571 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:54:13.571 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:54:13.571 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:54:13.574 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:54:13.574 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:54:13.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:13.587+0000 7f0681c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:13.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:13.602+0000 7f0681c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:14.428 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'unique_pool_5' created 2024-04-23T01:54:14.446 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_5 rados --yes-i-really-mean-it || true 2024-04-23T01:54:14.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:14.622+0000 7fcf16e24640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:14.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:14.637+0000 7fcf16e24640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:14.870 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:54:14.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:14.973+0000 7f8aa202d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:14.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:14.978+0000 7f8aa202d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:15.428 INFO:teuthology.orchestra.run.smithi090.stderr:enabled application 'rados' on pool 'unique_pool_5' 2024-04-23T01:54:16.452 DEBUG:teuthology.orchestra.run.smithi090:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 8192 --object-size 65536 -p unique_pool_5 bench 90 write' 2024-04-23T01:54:16.455 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-04-23T01:54:16.455 INFO:teuthology.orchestra.run:waiting for 3000 2024-04-23T01:54:16.518 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:hints = 1 2024-04-23T01:54:16.518 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Maintaining 16 concurrent writes of 8192 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-04-23T01:54:16.519 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Object prefix: benchmark_data_smithi090_50289 2024-04-23T01:54:16.519 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:54:16.519 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 0 0 0 0 0 0 - 0 2024-04-23T01:54:17.074 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:54:17.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:17.197+0000 7f216935d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:17.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:17.206+0000 7f216935d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:17.518 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 1 16 136 120 0.937444 0.9375 0.113849 0.124581 2024-04-23T01:54:18.519 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 2 16 281 265 1.03507 1.13281 0.159991 0.116368 2024-04-23T01:54:18.574 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:54:18.575 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-23T01:54:18.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:18.710+0000 7f2396a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:18.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:18.723+0000 7f2396a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:19.400 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:54:19.444 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.0 to 1 (10000) 2024-04-23T01:54:19.466 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-23T01:54:19.519 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 3 16 432 416 1.08324 1.17969 0.109687 0.111372 2024-04-23T01:54:19.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:19.546+0000 7f6f0646d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:19.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:19.553+0000 7f6f0646d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:19.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:19.635+0000 7f102c554640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:19.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:19.649+0000 7f102c554640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:20.444 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.1 to 1 (10000) 2024-04-23T01:54:20.460 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-23T01:54:20.519 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 4 16 576 560 1.09365 1.125 0.0689723 0.111754 2024-04-23T01:54:20.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:20.592+0000 7f56a155d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:20.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:20.606+0000 7f56a155d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:21.449 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.2 to 1 (10000) 2024-04-23T01:54:21.466 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-23T01:54:21.519 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 5 16 716 700 1.09365 1.09375 0.0272203 0.112363 2024-04-23T01:54:21.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:21.583+0000 7f21a7ced640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:21.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:21.594+0000 7f21a7ced640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:21.798 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:54:21.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:21.926+0000 7fa03c6ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:21.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:21.932+0000 7fa03c6ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:22.455 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.3 to 1 (10000) 2024-04-23T01:54:22.473 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-23T01:54:22.473 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:54:22.519 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 6 16 858 842 1.09626 1.10938 0.114833 0.112409 2024-04-23T01:54:22.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:22.616+0000 7f56a6e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:22.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:22.629+0000 7f56a6e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:22.855 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:54:22.856 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:54:22.871 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-04-23T01:54:22.455023+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6562336,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":866,"num_write_kb":7726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":813,"ondisk_log_size":813,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":371472,"kb_used_data":10160,"kb_used_omap":0,"kb_used_meta":361280,"kb_avail":377115888,"statfs":{"total":386547056640,"available":386166669312,"internally_reserved":0,"allocated":10403840,"data_stored":19179485,"data_compressed":952519,"data_compressed_allocated":8499200,"data_compressed_original":18472960,"omap_allocated":0,"internal_metadata":369950720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5906432,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":721,"num_write_kb":5768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.800665"},"pg_stats":[{"pgid":"8.7","version":"52'32","reported_seq":49,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-23T01:54:21.532738+0000","last_change":"2024-04-23T01:54:15.581616+0000","last_active":"2024-04-23T01:54:21.532738+0000","last_peered":"2024-04-23T01:54:21.532738+0000","last_clean":"2024-04-23T01:54:21.532738+0000","last_became_active":"2024-04-23T01:54:15.576738+0000","last_became_peered":"2024-04-23T01:54:15.576738+0000","last_unstale":"2024-04-23T01:54:21.532738+0000","last_undegraded":"2024-04-23T01:54:21.532738+0000","last_fullsized":"2024-04-23T01:54:21.532738+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"52'40","reported_seq":57,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-23T01:54:21.504833+0000","last_change":"2024-04-23T01:54:15.596716+0000","last_active":"2024-04-23T01:54:21.504833+0000","last_peered":"2024-04-23T01:54:21.504833+0000","last_clean":"2024-04-23T01:54:21.504833+0000","last_became_active":"2024-04-23T01:54:15.589907+0000","last_became_peered":"2024-04-23T01:54:15.589907+0000","last_unstale":"2024-04-23T01:54:21.504833+0000","last_undegraded":"2024-04-23T01:54:21.504833+0000","last_fullsized":"2024-04-23T01:54:21.504833+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"54'31","reported_seq":48,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-23T01:54:21.361522+0000","last_change":"2024-04-23T01:54:15.700282+0000","last_active":"2024-04-23T01:54:21.361522+0000","last_peered":"2024-04-23T01:54:21.361522+0000","last_clean":"2024-04-23T01:54:21.361522+0000","last_became_active":"2024-04-23T01:54:15.686441+0000","last_became_peered":"2024-04-23T01:54:15.686441+0000","last_unstale":"2024-04-23T01:54:21.361522+0000","last_undegraded":"2024-04-23T01:54:21.361522+0000","last_fullsized":"2024-04-23T01:54:21.361522+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"53'55","reported_seq":71,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T01:54:21.324199+0000","last_change":"2024-04-23T01:54:15.681870+0000","last_active":"2024-04-23T01:54:21.324199+0000","last_peered":"2024-04-23T01:54:21.324199+0000","last_clean":"2024-04-23T01:54:21.324199+0000","last_became_active":"2024-04-23T01:54:15.677678+0000","last_became_peered":"2024-04-23T01:54:15.677678+0000","last_unstale":"2024-04-23T01:54:21.324199+0000","last_undegraded":"2024-04-23T01:54:21.324199+0000","last_fullsized":"2024-04-23T01:54:21.324199+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458752,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3","version":"54'31","reported_seq":48,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-23T01:54:21.604034+0000","last_change":"2024-04-23T01:54:15.711951+0000","last_active":"2024-04-23T01:54:21.604034+0000","last_peered":"2024-04-23T01:54:21.604034+0000","last_clean":"2024-04-23T01:54:21.604034+0000","last_became_active":"2024-04-23T01:54:15.696611+0000","last_became_peered":"2024-04-23T01:54:15.696611+0000","last_unstale":"2024-04-23T01:54:21.604034+0000","last_undegraded":"2024-04-23T01:54:21.604034+0000","last_fullsized":"2024-04-23T01:54:21.604034+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"53'23","reported_seq":39,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T01:54:20.933258+0000","last_change":"2024-04-23T01:54:15.676654+0000","last_active":"2024-04-23T01:54:20.933258+0000","last_peered":"2024-04-23T01:54:20.933258+0000","last_clean":"2024-04-23T01:54:20.933258+0000","last_became_active":"2024-04-23T01:54:15.672239+0000","last_became_peered":"2024-04-23T01:54:15.672239+0000","last_unstale":"2024-04-23T01:54:20.933258+0000","last_undegraded":"2024-04-23T01:54:20.933258+0000","last_fullsized":"2024-04-23T01:54:20.933258+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"54'30","reported_seq":47,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-23T01:54:21.700889+0000","last_change":"2024-04-23T01:54:15.609601+0000","last_active":"2024-04-23T01:54:21.700889+0000","last_peered":"2024-04-23T01:54:21.700889+0000","last_clean":"2024-04-23T01:54:21.700889+0000","last_became_active":"2024-04-23T01:54:15.598424+0000","last_became_peered":"2024-04-23T01:54:15.598424+0000","last_unstale":"2024-04-23T01:54:21.700889+0000","last_undegraded":"2024-04-23T01:54:21.700889+0000","last_fullsized":"2024-04-23T01:54:21.700889+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":253952,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"54'79","reported_seq":96,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-23T01:54:21.260062+0000","last_change":"2024-04-23T01:54:15.718344+0000","last_active":"2024-04-23T01:54:21.260062+0000","last_peered":"2024-04-23T01:54:21.260062+0000","last_clean":"2024-04-23T01:54:21.260062+0000","last_became_active":"2024-04-23T01:54:15.706071+0000","last_became_peered":"2024-04-23T01:54:15.706071+0000","last_unstale":"2024-04-23T01:54:21.260062+0000","last_undegraded":"2024-04-23T01:54:21.260062+0000","last_fullsized":"2024-04-23T01:54:21.260062+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"53'39","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T01:54:21.065914+0000","last_change":"2024-04-23T01:54:15.596696+0000","last_active":"2024-04-23T01:54:21.065914+0000","last_peered":"2024-04-23T01:54:21.065914+0000","last_clean":"2024-04-23T01:54:21.065914+0000","last_became_active":"2024-04-23T01:54:15.588060+0000","last_became_peered":"2024-04-23T01:54:15.588060+0000","last_unstale":"2024-04-23T01:54:21.065914+0000","last_undegraded":"2024-04-23T01:54:21.065914+0000","last_fullsized":"2024-04-23T01:54:21.065914+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":251,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-23T01:54:20.854741+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:54:20.854741+0000","last_peered":"2024-04-23T01:54:20.854741+0000","last_clean":"2024-04-23T01:54:20.854741+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:54:20.854741+0000","last_undegraded":"2024-04-23T01:54:20.854741+0000","last_fullsized":"2024-04-23T01:54:20.854741+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":122,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-23T01:54:20.506871+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:54:20.506871+0000","last_peered":"2024-04-23T01:54:20.506871+0000","last_clean":"2024-04-23T01:54:20.506871+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:54:20.506871+0000","last_undegraded":"2024-04-23T01:54:20.506871+0000","last_fullsized":"2024-04-23T01:54:20.506871+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"54'63","reported_seq":80,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-23T01:54:21.638111+0000","last_change":"2024-04-23T01:54:15.705667+0000","last_active":"2024-04-23T01:54:21.638111+0000","last_peered":"2024-04-23T01:54:21.638111+0000","last_clean":"2024-04-23T01:54:21.638111+0000","last_became_active":"2024-04-23T01:54:15.701510+0000","last_became_peered":"2024-04-23T01:54:15.701510+0000","last_unstale":"2024-04-23T01:54:21.638111+0000","last_undegraded":"2024-04-23T01:54:21.638111+0000","last_fullsized":"2024-04-23T01:54:21.638111+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T01:54:19.500270+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:54:19.500270+0000","last_peered":"2024-04-23T01:54:19.500270+0000","last_clean":"2024-04-23T01:54:19.500270+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:54:19.500270+0000","last_undegraded":"2024-04-23T01:54:19.500270+0000","last_fullsized":"2024-04-23T01:54:19.500270+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"53'71","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T01:54:20.055567+0000","last_change":"2024-04-23T01:54:15.668120+0000","last_active":"2024-04-23T01:54:20.055567+0000","last_peered":"2024-04-23T01:54:20.055567+0000","last_clean":"2024-04-23T01:54:20.055567+0000","last_became_active":"2024-04-23T01:54:15.661561+0000","last_became_peered":"2024-04-23T01:54:15.661561+0000","last_unstale":"2024-04-23T01:54:20.055567+0000","last_undegraded":"2024-04-23T01:54:20.055567+0000","last_fullsized":"2024-04-23T01:54:20.055567+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":122,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-23T01:54:20.497507+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:54:20.497507+0000","last_peered":"2024-04-23T01:54:20.497507+0000","last_clean":"2024-04-23T01:54:20.497507+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:54:20.497507+0000","last_undegraded":"2024-04-23T01:54:20.497507+0000","last_fullsized":"2024-04-23T01:54:20.497507+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"54'39","reported_seq":56,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-23T01:54:21.147551+0000","last_change":"2024-04-23T01:54:15.677434+0000","last_active":"2024-04-23T01:54:21.147551+0000","last_peered":"2024-04-23T01:54:21.147551+0000","last_clean":"2024-04-23T01:54:21.147551+0000","last_became_active":"2024-04-23T01:54:15.670406+0000","last_became_peered":"2024-04-23T01:54:15.670406+0000","last_unstale":"2024-04-23T01:54:21.147551+0000","last_undegraded":"2024-04-23T01:54:21.147551+0000","last_fullsized":"2024-04-23T01:54:21.147551+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T01:54:19.516473+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:54:19.516473+0000","last_peered":"2024-04-23T01:54:19.516473+0000","last_clean":"2024-04-23T01:54:19.516473+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:54:19.516473+0000","last_undegraded":"2024-04-23T01:54:19.516473+0000","last_fullsized":"2024-04-23T01:54:19.516473+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.e","version":"52'40","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T01:54:19.505838+0000","last_change":"2024-04-23T01:54:15.652153+0000","last_active":"2024-04-23T01:54:19.505838+0000","last_peered":"2024-04-23T01:54:19.505838+0000","last_clean":"2024-04-23T01:54:19.505838+0000","last_became_active":"2024-04-23T01:54:15.644332+0000","last_became_peered":"2024-04-23T01:54:15.644332+0000","last_unstale":"2024-04-23T01:54:19.505838+0000","last_undegraded":"2024-04-23T01:54:19.505838+0000","last_fullsized":"2024-04-23T01:54:19.505838+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":123,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-23T01:54:21.515506+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:54:21.515506+0000","last_peered":"2024-04-23T01:54:21.515506+0000","last_clean":"2024-04-23T01:54:21.515506+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:54:21.515506+0000","last_undegraded":"2024-04-23T01:54:21.515506+0000","last_fullsized":"2024-04-23T01:54:21.515506+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"54'63","reported_seq":80,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-23T01:54:20.924718+0000","last_change":"2024-04-23T01:54:15.637842+0000","last_active":"2024-04-23T01:54:20.924718+0000","last_peered":"2024-04-23T01:54:20.924718+0000","last_clean":"2024-04-23T01:54:20.924718+0000","last_became_active":"2024-04-23T01:54:15.631530+0000","last_became_peered":"2024-04-23T01:54:15.631530+0000","last_unstale":"2024-04-23T01:54:20.924718+0000","last_undegraded":"2024-04-23T01:54:20.924718+0000","last_fullsized":"2024-04-23T01:54:20.924718+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":118,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-23T01:54:20.863538+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:54:20.863538+0000","last_peered":"2024-04-23T01:54:20.863538+0000","last_clean":"2024-04-23T01:54:20.863538+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:54:20.863538+0000","last_undegraded":"2024-04-23T01:54:20.863538+0000","last_fullsized":"2024-04-23T01:54:20.863538+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"53'39","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T01:54:20.649139+0000","last_change":"2024-04-23T01:54:15.617410+0000","last_active":"2024-04-23T01:54:20.649139+0000","last_peered":"2024-04-23T01:54:20.649139+0000","last_clean":"2024-04-23T01:54:20.649139+0000","last_became_active":"2024-04-23T01:54:15.610208+0000","last_became_peered":"2024-04-23T01:54:15.610208+0000","last_unstale":"2024-04-23T01:54:20.649139+0000","last_undegraded":"2024-04-23T01:54:20.649139+0000","last_fullsized":"2024-04-23T01:54:20.649139+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":123,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-23T01:54:21.525523+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:54:21.525523+0000","last_peered":"2024-04-23T01:54:21.525523+0000","last_clean":"2024-04-23T01:54:21.525523+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:54:21.525523+0000","last_undegraded":"2024-04-23T01:54:21.525523+0000","last_fullsized":"2024-04-23T01:54:21.525523+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"53'41","reported_seq":57,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T01:54:21.608682+0000","last_change":"2024-04-23T01:54:15.630052+0000","last_active":"2024-04-23T01:54:21.608682+0000","last_peered":"2024-04-23T01:54:21.608682+0000","last_clean":"2024-04-23T01:54:21.608682+0000","last_became_active":"2024-04-23T01:54:15.624615+0000","last_became_peered":"2024-04-23T01:54:15.624615+0000","last_unstale":"2024-04-23T01:54:21.608682+0000","last_undegraded":"2024-04-23T01:54:21.608682+0000","last_fullsized":"2024-04-23T01:54:21.608682+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":344064,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":122,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-23T01:54:20.508635+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:54:20.508635+0000","last_peered":"2024-04-23T01:54:20.508635+0000","last_clean":"2024-04-23T01:54:20.508635+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:54:20.508635+0000","last_undegraded":"2024-04-23T01:54:20.508635+0000","last_fullsized":"2024-04-23T01:54:20.508635+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":5971968,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":729,"num_write_kb":5832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8945664,"data_stored":17891328,"data_compressed":921318,"data_compressed_allocated":8945664,"data_compressed_original":17891328,"omap_allocated":0,"internal_metadata":0},"log_size":716,"ondisk_log_size":716,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706218,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":91788,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":89280,"kb_avail":94280052,"statfs":{"total":96636764160,"available":96542773248,"internally_reserved":0,"allocated":2560000,"data_stored":4819969,"data_compressed":239191,"data_compressed_allocated":2072576,"data_compressed_original":4636672,"omap_allocated":0,"internal_metadata":91422720},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706218,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":91332,"kb_used_data":2044,"kb_used_omap":0,"kb_used_meta":89280,"kb_avail":94280508,"statfs":{"total":96636764160,"available":96543240192,"internally_reserved":0,"allocated":2093056,"data_stored":3912359,"data_compressed":192801,"data_compressed_allocated":1622016,"data_compressed_original":3735552,"omap_allocated":0,"internal_metadata":91422720},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706218,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":96380,"kb_used_data":2932,"kb_used_omap":0,"kb_used_meta":93440,"kb_avail":94275460,"statfs":{"total":96636764160,"available":96538071040,"internally_reserved":0,"allocated":3002368,"data_stored":5237217,"data_compressed":261108,"data_compressed_allocated":2535424,"data_compressed_original":5070848,"omap_allocated":0,"internal_metadata":95682560},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706218,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":91972,"kb_used_data":2684,"kb_used_omap":0,"kb_used_meta":89280,"kb_avail":94279868,"statfs":{"total":96636764160,"available":96542584832,"internally_reserved":0,"allocated":2748416,"data_stored":5209940,"data_compressed":259419,"data_compressed_allocated":2269184,"data_compressed_original":5029888,"omap_allocated":0,"internal_metadata":91422720},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2310144,"data_stored":4620288,"data_compressed":237906,"data_compressed_allocated":2310144,"data_compressed_original":4620288,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2711552,"data_stored":5423104,"data_compressed":279254,"data_compressed_allocated":2711552,"data_compressed_original":5423104,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1695744,"data_stored":3391488,"data_compressed":174664,"data_compressed_allocated":1695744,"data_compressed_original":3391488,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2228224,"data_stored":4456448,"data_compressed":229494,"data_compressed_allocated":2228224,"data_compressed_original":4456448,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:54:22.872 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:54:22.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:22.992+0000 7f1e9535d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:23.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:23.002+0000 7f1e9535d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:23.235 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:54:23.235 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:54:23.251 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-04-23T01:54:22.455023+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6562336,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":866,"num_write_kb":7726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":813,"ondisk_log_size":813,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":371472,"kb_used_data":10160,"kb_used_omap":0,"kb_used_meta":361280,"kb_avail":377115888,"statfs":{"total":386547056640,"available":386166669312,"internally_reserved":0,"allocated":10403840,"data_stored":19179485,"data_compressed":952519,"data_compressed_allocated":8499200,"data_compressed_original":18472960,"omap_allocated":0,"internal_metadata":369950720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5906432,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":721,"num_write_kb":5768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.800665"},"pg_stats":[{"pgid":"8.7","version":"52'32","reported_seq":49,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-23T01:54:21.532738+0000","last_change":"2024-04-23T01:54:15.581616+0000","last_active":"2024-04-23T01:54:21.532738+0000","last_peered":"2024-04-23T01:54:21.532738+0000","last_clean":"2024-04-23T01:54:21.532738+0000","last_became_active":"2024-04-23T01:54:15.576738+0000","last_became_peered":"2024-04-23T01:54:15.576738+0000","last_unstale":"2024-04-23T01:54:21.532738+0000","last_undegraded":"2024-04-23T01:54:21.532738+0000","last_fullsized":"2024-04-23T01:54:21.532738+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"52'40","reported_seq":57,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-23T01:54:21.504833+0000","last_change":"2024-04-23T01:54:15.596716+0000","last_active":"2024-04-23T01:54:21.504833+0000","last_peered":"2024-04-23T01:54:21.504833+0000","last_clean":"2024-04-23T01:54:21.504833+0000","last_became_active":"2024-04-23T01:54:15.589907+0000","last_became_peered":"2024-04-23T01:54:15.589907+0000","last_unstale":"2024-04-23T01:54:21.504833+0000","last_undegraded":"2024-04-23T01:54:21.504833+0000","last_fullsized":"2024-04-23T01:54:21.504833+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"54'31","reported_seq":48,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-23T01:54:21.361522+0000","last_change":"2024-04-23T01:54:15.700282+0000","last_active":"2024-04-23T01:54:21.361522+0000","last_peered":"2024-04-23T01:54:21.361522+0000","last_clean":"2024-04-23T01:54:21.361522+0000","last_became_active":"2024-04-23T01:54:15.686441+0000","last_became_peered":"2024-04-23T01:54:15.686441+0000","last_unstale":"2024-04-23T01:54:21.361522+0000","last_undegraded":"2024-04-23T01:54:21.361522+0000","last_fullsized":"2024-04-23T01:54:21.361522+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"53'55","reported_seq":71,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T01:54:21.324199+0000","last_change":"2024-04-23T01:54:15.681870+0000","last_active":"2024-04-23T01:54:21.324199+0000","last_peered":"2024-04-23T01:54:21.324199+0000","last_clean":"2024-04-23T01:54:21.324199+0000","last_became_active":"2024-04-23T01:54:15.677678+0000","last_became_peered":"2024-04-23T01:54:15.677678+0000","last_unstale":"2024-04-23T01:54:21.324199+0000","last_undegraded":"2024-04-23T01:54:21.324199+0000","last_fullsized":"2024-04-23T01:54:21.324199+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458752,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3","version":"54'31","reported_seq":48,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-23T01:54:21.604034+0000","last_change":"2024-04-23T01:54:15.711951+0000","last_active":"2024-04-23T01:54:21.604034+0000","last_peered":"2024-04-23T01:54:21.604034+0000","last_clean":"2024-04-23T01:54:21.604034+0000","last_became_active":"2024-04-23T01:54:15.696611+0000","last_became_peered":"2024-04-23T01:54:15.696611+0000","last_unstale":"2024-04-23T01:54:21.604034+0000","last_undegraded":"2024-04-23T01:54:21.604034+0000","last_fullsized":"2024-04-23T01:54:21.604034+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"53'23","reported_seq":39,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T01:54:20.933258+0000","last_change":"2024-04-23T01:54:15.676654+0000","last_active":"2024-04-23T01:54:20.933258+0000","last_peered":"2024-04-23T01:54:20.933258+0000","last_clean":"2024-04-23T01:54:20.933258+0000","last_became_active":"2024-04-23T01:54:15.672239+0000","last_became_peered":"2024-04-23T01:54:15.672239+0000","last_unstale":"2024-04-23T01:54:20.933258+0000","last_undegraded":"2024-04-23T01:54:20.933258+0000","last_fullsized":"2024-04-23T01:54:20.933258+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"54'30","reported_seq":47,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-23T01:54:21.700889+0000","last_change":"2024-04-23T01:54:15.609601+0000","last_active":"2024-04-23T01:54:21.700889+0000","last_peered":"2024-04-23T01:54:21.700889+0000","last_clean":"2024-04-23T01:54:21.700889+0000","last_became_active":"2024-04-23T01:54:15.598424+0000","last_became_peered":"2024-04-23T01:54:15.598424+0000","last_unstale":"2024-04-23T01:54:21.700889+0000","last_undegraded":"2024-04-23T01:54:21.700889+0000","last_fullsized":"2024-04-23T01:54:21.700889+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":253952,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"54'79","reported_seq":96,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-23T01:54:21.260062+0000","last_change":"2024-04-23T01:54:15.718344+0000","last_active":"2024-04-23T01:54:21.260062+0000","last_peered":"2024-04-23T01:54:21.260062+0000","last_clean":"2024-04-23T01:54:21.260062+0000","last_became_active":"2024-04-23T01:54:15.706071+0000","last_became_peered":"2024-04-23T01:54:15.706071+0000","last_unstale":"2024-04-23T01:54:21.260062+0000","last_undegraded":"2024-04-23T01:54:21.260062+0000","last_fullsized":"2024-04-23T01:54:21.260062+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"53'39","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T01:54:21.065914+0000","last_change":"2024-04-23T01:54:15.596696+0000","last_active":"2024-04-23T01:54:21.065914+0000","last_peered":"2024-04-23T01:54:21.065914+0000","last_clean":"2024-04-23T01:54:21.065914+0000","last_became_active":"2024-04-23T01:54:15.588060+0000","last_became_peered":"2024-04-23T01:54:15.588060+0000","last_unstale":"2024-04-23T01:54:21.065914+0000","last_undegraded":"2024-04-23T01:54:21.065914+0000","last_fullsized":"2024-04-23T01:54:21.065914+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":251,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-23T01:54:20.854741+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:54:20.854741+0000","last_peered":"2024-04-23T01:54:20.854741+0000","last_clean":"2024-04-23T01:54:20.854741+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:54:20.854741+0000","last_undegraded":"2024-04-23T01:54:20.854741+0000","last_fullsized":"2024-04-23T01:54:20.854741+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":122,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-23T01:54:20.506871+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:54:20.506871+0000","last_peered":"2024-04-23T01:54:20.506871+0000","last_clean":"2024-04-23T01:54:20.506871+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:54:20.506871+0000","last_undegraded":"2024-04-23T01:54:20.506871+0000","last_fullsized":"2024-04-23T01:54:20.506871+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"54'63","reported_seq":80,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-23T01:54:21.638111+0000","last_change":"2024-04-23T01:54:15.705667+0000","last_active":"2024-04-23T01:54:21.638111+0000","last_peered":"2024-04-23T01:54:21.638111+0000","last_clean":"2024-04-23T01:54:21.638111+0000","last_became_active":"2024-04-23T01:54:15.701510+0000","last_became_peered":"2024-04-23T01:54:15.701510+0000","last_unstale":"2024-04-23T01:54:21.638111+0000","last_undegraded":"2024-04-23T01:54:21.638111+0000","last_fullsized":"2024-04-23T01:54:21.638111+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T01:54:19.500270+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:54:19.500270+0000","last_peered":"2024-04-23T01:54:19.500270+0000","last_clean":"2024-04-23T01:54:19.500270+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:54:19.500270+0000","last_undegraded":"2024-04-23T01:54:19.500270+0000","last_fullsized":"2024-04-23T01:54:19.500270+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"53'71","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T01:54:20.055567+0000","last_change":"2024-04-23T01:54:15.668120+0000","last_active":"2024-04-23T01:54:20.055567+0000","last_peered":"2024-04-23T01:54:20.055567+0000","last_clean":"2024-04-23T01:54:20.055567+0000","last_became_active":"2024-04-23T01:54:15.661561+0000","last_became_peered":"2024-04-23T01:54:15.661561+0000","last_unstale":"2024-04-23T01:54:20.055567+0000","last_undegraded":"2024-04-23T01:54:20.055567+0000","last_fullsized":"2024-04-23T01:54:20.055567+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":122,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-23T01:54:20.497507+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:54:20.497507+0000","last_peered":"2024-04-23T01:54:20.497507+0000","last_clean":"2024-04-23T01:54:20.497507+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:54:20.497507+0000","last_undegraded":"2024-04-23T01:54:20.497507+0000","last_fullsized":"2024-04-23T01:54:20.497507+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"54'39","reported_seq":56,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-23T01:54:21.147551+0000","last_change":"2024-04-23T01:54:15.677434+0000","last_active":"2024-04-23T01:54:21.147551+0000","last_peered":"2024-04-23T01:54:21.147551+0000","last_clean":"2024-04-23T01:54:21.147551+0000","last_became_active":"2024-04-23T01:54:15.670406+0000","last_became_peered":"2024-04-23T01:54:15.670406+0000","last_unstale":"2024-04-23T01:54:21.147551+0000","last_undegraded":"2024-04-23T01:54:21.147551+0000","last_fullsized":"2024-04-23T01:54:21.147551+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T01:54:19.516473+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:54:19.516473+0000","last_peered":"2024-04-23T01:54:19.516473+0000","last_clean":"2024-04-23T01:54:19.516473+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:54:19.516473+0000","last_undegraded":"2024-04-23T01:54:19.516473+0000","last_fullsized":"2024-04-23T01:54:19.516473+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.e","version":"52'40","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T01:54:19.505838+0000","last_change":"2024-04-23T01:54:15.652153+0000","last_active":"2024-04-23T01:54:19.505838+0000","last_peered":"2024-04-23T01:54:19.505838+0000","last_clean":"2024-04-23T01:54:19.505838+0000","last_became_active":"2024-04-23T01:54:15.644332+0000","last_became_peered":"2024-04-23T01:54:15.644332+0000","last_unstale":"2024-04-23T01:54:19.505838+0000","last_undegraded":"2024-04-23T01:54:19.505838+0000","last_fullsized":"2024-04-23T01:54:19.505838+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":123,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-23T01:54:21.515506+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:54:21.515506+0000","last_peered":"2024-04-23T01:54:21.515506+0000","last_clean":"2024-04-23T01:54:21.515506+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:54:21.515506+0000","last_undegraded":"2024-04-23T01:54:21.515506+0000","last_fullsized":"2024-04-23T01:54:21.515506+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"54'63","reported_seq":80,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-23T01:54:20.924718+0000","last_change":"2024-04-23T01:54:15.637842+0000","last_active":"2024-04-23T01:54:20.924718+0000","last_peered":"2024-04-23T01:54:20.924718+0000","last_clean":"2024-04-23T01:54:20.924718+0000","last_became_active":"2024-04-23T01:54:15.631530+0000","last_became_peered":"2024-04-23T01:54:15.631530+0000","last_unstale":"2024-04-23T01:54:20.924718+0000","last_undegraded":"2024-04-23T01:54:20.924718+0000","last_fullsized":"2024-04-23T01:54:20.924718+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":118,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-23T01:54:20.863538+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:54:20.863538+0000","last_peered":"2024-04-23T01:54:20.863538+0000","last_clean":"2024-04-23T01:54:20.863538+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:54:20.863538+0000","last_undegraded":"2024-04-23T01:54:20.863538+0000","last_fullsized":"2024-04-23T01:54:20.863538+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"53'39","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T01:54:20.649139+0000","last_change":"2024-04-23T01:54:15.617410+0000","last_active":"2024-04-23T01:54:20.649139+0000","last_peered":"2024-04-23T01:54:20.649139+0000","last_clean":"2024-04-23T01:54:20.649139+0000","last_became_active":"2024-04-23T01:54:15.610208+0000","last_became_peered":"2024-04-23T01:54:15.610208+0000","last_unstale":"2024-04-23T01:54:20.649139+0000","last_undegraded":"2024-04-23T01:54:20.649139+0000","last_fullsized":"2024-04-23T01:54:20.649139+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":123,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-23T01:54:21.525523+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:54:21.525523+0000","last_peered":"2024-04-23T01:54:21.525523+0000","last_clean":"2024-04-23T01:54:21.525523+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:54:21.525523+0000","last_undegraded":"2024-04-23T01:54:21.525523+0000","last_fullsized":"2024-04-23T01:54:21.525523+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"53'41","reported_seq":57,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T01:54:21.608682+0000","last_change":"2024-04-23T01:54:15.630052+0000","last_active":"2024-04-23T01:54:21.608682+0000","last_peered":"2024-04-23T01:54:21.608682+0000","last_clean":"2024-04-23T01:54:21.608682+0000","last_became_active":"2024-04-23T01:54:15.624615+0000","last_became_peered":"2024-04-23T01:54:15.624615+0000","last_unstale":"2024-04-23T01:54:21.608682+0000","last_undegraded":"2024-04-23T01:54:21.608682+0000","last_fullsized":"2024-04-23T01:54:21.608682+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":344064,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":122,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-23T01:54:20.508635+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:54:20.508635+0000","last_peered":"2024-04-23T01:54:20.508635+0000","last_clean":"2024-04-23T01:54:20.508635+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:54:20.508635+0000","last_undegraded":"2024-04-23T01:54:20.508635+0000","last_fullsized":"2024-04-23T01:54:20.508635+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":5971968,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":729,"num_write_kb":5832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8945664,"data_stored":17891328,"data_compressed":921318,"data_compressed_allocated":8945664,"data_compressed_original":17891328,"omap_allocated":0,"internal_metadata":0},"log_size":716,"ondisk_log_size":716,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706218,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":91788,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":89280,"kb_avail":94280052,"statfs":{"total":96636764160,"available":96542773248,"internally_reserved":0,"allocated":2560000,"data_stored":4819969,"data_compressed":239191,"data_compressed_allocated":2072576,"data_compressed_original":4636672,"omap_allocated":0,"internal_metadata":91422720},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706218,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":91332,"kb_used_data":2044,"kb_used_omap":0,"kb_used_meta":89280,"kb_avail":94280508,"statfs":{"total":96636764160,"available":96543240192,"internally_reserved":0,"allocated":2093056,"data_stored":3912359,"data_compressed":192801,"data_compressed_allocated":1622016,"data_compressed_original":3735552,"omap_allocated":0,"internal_metadata":91422720},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706218,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":96380,"kb_used_data":2932,"kb_used_omap":0,"kb_used_meta":93440,"kb_avail":94275460,"statfs":{"total":96636764160,"available":96538071040,"internally_reserved":0,"allocated":3002368,"data_stored":5237217,"data_compressed":261108,"data_compressed_allocated":2535424,"data_compressed_original":5070848,"omap_allocated":0,"internal_metadata":95682560},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706218,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":91972,"kb_used_data":2684,"kb_used_omap":0,"kb_used_meta":89280,"kb_avail":94279868,"statfs":{"total":96636764160,"available":96542584832,"internally_reserved":0,"allocated":2748416,"data_stored":5209940,"data_compressed":259419,"data_compressed_allocated":2269184,"data_compressed_original":5029888,"omap_allocated":0,"internal_metadata":91422720},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2310144,"data_stored":4620288,"data_compressed":237906,"data_compressed_allocated":2310144,"data_compressed_original":4620288,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2711552,"data_stored":5423104,"data_compressed":279254,"data_compressed_allocated":2711552,"data_compressed_original":5423104,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1695744,"data_stored":3391488,"data_compressed":174664,"data_compressed_allocated":1695744,"data_compressed_original":3391488,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2228224,"data_stored":4456448,"data_compressed":229494,"data_compressed_allocated":2228224,"data_compressed_original":4456448,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:54:23.252 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-23T01:54:23.253 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:54:23.253 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:54:23.253 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:54:23.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:23.399+0000 7f2ef735d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:23.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:23.410+0000 7f2ef735d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:23.440 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:54:23.440 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:54:23.440 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:54:23.440 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:54:23.440 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:54:23.441 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:54:23.441 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:54:23.441 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:54:23.441 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:54:23.441 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:54:23.441 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:54:23.441 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:54:23.444 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:54:23.444 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:54:23.444 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:54:23.519 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 7 16 989 973 1.08584 1.02344 0.030493 0.113869 2024-04-23T01:54:23.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:23.577+0000 7f138b95d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:23.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:23.588+0000 7f138b95d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:23.663 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:54:23.663 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:54:23.663 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:54:23.663 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:54:23.663 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:54:23.663 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:54:23.663 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:54:23.663 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:54:23.663 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:54:23.663 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:54:23.663 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:54:23.663 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:54:23.666 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:54:23.666 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:54:23.667 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:54:23.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:23.805+0000 7fa85ce2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:23.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:23.817+0000 7fa85ce2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:23.842 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:54:23.842 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:54:23.842 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:54:23.842 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:54:23.842 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:54:23.842 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:54:23.842 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:54:23.842 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:54:23.842 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:54:23.842 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:54:23.843 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:54:23.843 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:54:23.845 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:54:23.845 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:54:23.845 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:54:23.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:23.972+0000 7f5d6df9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:23.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:23.987+0000 7f5d6df9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:24.006 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:54:24.006 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:54:24.006 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:54:24.007 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:54:24.007 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:54:24.007 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:54:24.007 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:54:24.007 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:54:24.007 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:54:24.007 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:54:24.007 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:54:24.007 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:54:24.010 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:54:24.010 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:54:24.144 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:54:24.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:24.259+0000 7f62b206d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:24.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:24.266+0000 7f62b206d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:24.519 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 8 16 1138 1122 1.09561 1.16406 0.199846 0.113198 2024-04-23T01:54:25.519 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 9 16 1294 1278 1.10928 1.21875 0.0315341 0.11176 2024-04-23T01:54:26.458 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:54:26.519 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 10 16 1446 1430 1.11709 1.1875 0.218184 0.11112 2024-04-23T01:54:26.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:26.585+0000 7fc4c9b9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:26.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:26.592+0000 7fc4c9b9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:27.519 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 11 16 1603 1587 1.12703 1.22656 0.110259 0.110398 2024-04-23T01:54:28.519 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 12 16 1768 1752 1.14052 1.28906 0.156314 0.10895 2024-04-23T01:54:28.832 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:54:28.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:28.958+0000 7f0c5942d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:28.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:28.965+0000 7f0c5942d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:29.011 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:54:29.012 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-23T01:54:29.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:29.138+0000 7f65466ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:29.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:29.155+0000 7f65466ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:29.519 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 13 16 1908 1892 1.13692 1.09375 0.0230978 0.109393 2024-04-23T01:54:30.213 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.0 to 1 (10000) 2024-04-23T01:54:30.231 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-23T01:54:30.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:30.352+0000 7fed20d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:30.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:30.365+0000 7fed20d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:30.519 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 14 16 2049 2033 1.13439 1.10156 0.12919 0.109734 2024-04-23T01:54:31.176 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:54:31.219 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.1 to 1 (10000) 2024-04-23T01:54:31.240 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-23T01:54:31.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:31.306+0000 7f9ed42ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:31.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:31.315+0000 7f9ed42ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:31.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:31.404+0000 7ff81555d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:31.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:31.417+0000 7ff81555d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:31.520 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 15 16 2190 2174 1.13219 1.10156 0.108335 0.109913 2024-04-23T01:54:32.220 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.2 to 1 (10000) 2024-04-23T01:54:32.238 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-23T01:54:32.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:32.354+0000 7f5e51a06640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:32.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:32.369+0000 7f5e51a06640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:32.520 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 16 16 2306 2290 1.11807 0.90625 0.0899893 0.111333 2024-04-23T01:54:33.224 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.3 to 1 (10000) 2024-04-23T01:54:33.241 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-23T01:54:33.242 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:54:33.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:33.367+0000 7f6a4b55d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:33.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:33.378+0000 7f6a4b55d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:33.520 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 17 16 2460 2444 1.12307 1.20312 0.051634 0.110864 2024-04-23T01:54:33.558 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:54:33.670 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:54:33.670 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:54:33.685 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2024-04-23T01:54:33.657446+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":18596384,"num_objects":278,"num_object_clones":0,"num_object_copies":834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":2335,"num_write_kb":19478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2291,"ondisk_log_size":2291,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":389236,"kb_used_data":27924,"kb_used_omap":0,"kb_used_meta":361280,"kb_avail":377098124,"statfs":{"total":386547056640,"available":386148478976,"internally_reserved":0,"allocated":28594176,"data_stored":55349933,"data_compressed":2816571,"data_compressed_allocated":26558464,"data_compressed_original":54591488,"omap_allocated":0,"internal_metadata":369950720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6070272,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":741,"num_write_kb":5928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000684"},"pg_stats":[{"pgid":"8.7","version":"58'183","reported_seq":210,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.480114+0000","last_change":"2024-04-23T01:54:15.581616+0000","last_active":"2024-04-23T01:54:31.480114+0000","last_peered":"2024-04-23T01:54:31.480114+0000","last_clean":"2024-04-23T01:54:31.480114+0000","last_became_active":"2024-04-23T01:54:15.576738+0000","last_became_peered":"2024-04-23T01:54:15.576738+0000","last_unstale":"2024-04-23T01:54:31.480114+0000","last_undegraded":"2024-04-23T01:54:31.480114+0000","last_fullsized":"2024-04-23T01:54:31.480114+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"58'111","reported_seq":138,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.699628+0000","last_change":"2024-04-23T01:54:15.596716+0000","last_active":"2024-04-23T01:54:31.699628+0000","last_peered":"2024-04-23T01:54:31.699628+0000","last_clean":"2024-04-23T01:54:31.699628+0000","last_became_active":"2024-04-23T01:54:15.589907+0000","last_became_peered":"2024-04-23T01:54:15.589907+0000","last_unstale":"2024-04-23T01:54:31.699628+0000","last_undegraded":"2024-04-23T01:54:31.699628+0000","last_fullsized":"2024-04-23T01:54:31.699628+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"58'97","reported_seq":124,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.617346+0000","last_change":"2024-04-23T01:54:15.700282+0000","last_active":"2024-04-23T01:54:31.617346+0000","last_peered":"2024-04-23T01:54:31.617346+0000","last_clean":"2024-04-23T01:54:31.617346+0000","last_became_active":"2024-04-23T01:54:15.686441+0000","last_became_peered":"2024-04-23T01:54:15.686441+0000","last_unstale":"2024-04-23T01:54:31.617346+0000","last_undegraded":"2024-04-23T01:54:31.617346+0000","last_fullsized":"2024-04-23T01:54:31.617346+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":802816,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98,"num_write_kb":784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"56'120","reported_seq":143,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:30.823497+0000","last_change":"2024-04-23T01:54:15.681870+0000","last_active":"2024-04-23T01:54:30.823497+0000","last_peered":"2024-04-23T01:54:30.823497+0000","last_clean":"2024-04-23T01:54:30.823497+0000","last_became_active":"2024-04-23T01:54:15.677678+0000","last_became_peered":"2024-04-23T01:54:15.677678+0000","last_unstale":"2024-04-23T01:54:30.823497+0000","last_undegraded":"2024-04-23T01:54:30.823497+0000","last_fullsized":"2024-04-23T01:54:30.823497+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3","version":"56'112","reported_seq":138,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.615576+0000","last_change":"2024-04-23T01:54:15.711951+0000","last_active":"2024-04-23T01:54:31.615576+0000","last_peered":"2024-04-23T01:54:31.615576+0000","last_clean":"2024-04-23T01:54:31.615576+0000","last_became_active":"2024-04-23T01:54:15.696611+0000","last_became_peered":"2024-04-23T01:54:15.696611+0000","last_unstale":"2024-04-23T01:54:31.615576+0000","last_undegraded":"2024-04-23T01:54:31.615576+0000","last_fullsized":"2024-04-23T01:54:31.615576+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"57'88","reported_seq":114,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.668306+0000","last_change":"2024-04-23T01:54:15.676654+0000","last_active":"2024-04-23T01:54:31.668306+0000","last_peered":"2024-04-23T01:54:31.668306+0000","last_clean":"2024-04-23T01:54:31.668306+0000","last_became_active":"2024-04-23T01:54:15.672239+0000","last_became_peered":"2024-04-23T01:54:15.672239+0000","last_unstale":"2024-04-23T01:54:31.668306+0000","last_undegraded":"2024-04-23T01:54:31.668306+0000","last_fullsized":"2024-04-23T01:54:31.668306+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":720896,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"58'172","reported_seq":199,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.670957+0000","last_change":"2024-04-23T01:54:15.609601+0000","last_active":"2024-04-23T01:54:31.670957+0000","last_peered":"2024-04-23T01:54:31.670957+0000","last_clean":"2024-04-23T01:54:31.670957+0000","last_became_active":"2024-04-23T01:54:15.598424+0000","last_became_peered":"2024-04-23T01:54:15.598424+0000","last_unstale":"2024-04-23T01:54:31.670957+0000","last_undegraded":"2024-04-23T01:54:31.670957+0000","last_fullsized":"2024-04-23T01:54:31.670957+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1417216,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":173,"num_write_kb":1384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"57'176","reported_seq":202,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.605634+0000","last_change":"2024-04-23T01:54:15.718344+0000","last_active":"2024-04-23T01:54:31.605634+0000","last_peered":"2024-04-23T01:54:31.605634+0000","last_clean":"2024-04-23T01:54:31.605634+0000","last_became_active":"2024-04-23T01:54:15.706071+0000","last_became_peered":"2024-04-23T01:54:15.706071+0000","last_unstale":"2024-04-23T01:54:31.605634+0000","last_undegraded":"2024-04-23T01:54:31.605634+0000","last_fullsized":"2024-04-23T01:54:31.605634+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"57'167","reported_seq":191,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.595726+0000","last_change":"2024-04-23T01:54:15.596696+0000","last_active":"2024-04-23T01:54:31.595726+0000","last_peered":"2024-04-23T01:54:31.595726+0000","last_clean":"2024-04-23T01:54:31.595726+0000","last_became_active":"2024-04-23T01:54:15.588060+0000","last_became_peered":"2024-04-23T01:54:15.588060+0000","last_unstale":"2024-04-23T01:54:31.595726+0000","last_undegraded":"2024-04-23T01:54:31.595726+0000","last_fullsized":"2024-04-23T01:54:31.595726+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1368064,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":167,"num_write_kb":1336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":261,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.616318+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:54:31.616318+0000","last_peered":"2024-04-23T01:54:31.616318+0000","last_clean":"2024-04-23T01:54:31.616318+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:54:31.616318+0000","last_undegraded":"2024-04-23T01:54:31.616318+0000","last_fullsized":"2024-04-23T01:54:31.616318+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":132,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.621120+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:54:31.621120+0000","last_peered":"2024-04-23T01:54:31.621120+0000","last_clean":"2024-04-23T01:54:31.621120+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:54:31.621120+0000","last_undegraded":"2024-04-23T01:54:31.621120+0000","last_fullsized":"2024-04-23T01:54:31.621120+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"57'168","reported_seq":194,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.633231+0000","last_change":"2024-04-23T01:54:15.705667+0000","last_active":"2024-04-23T01:54:31.633231+0000","last_peered":"2024-04-23T01:54:31.633231+0000","last_clean":"2024-04-23T01:54:31.633231+0000","last_became_active":"2024-04-23T01:54:15.701510+0000","last_became_peered":"2024-04-23T01:54:15.701510+0000","last_unstale":"2024-04-23T01:54:31.633231+0000","last_undegraded":"2024-04-23T01:54:31.633231+0000","last_fullsized":"2024-04-23T01:54:31.633231+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":1344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":123,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:30.813603+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:54:30.813603+0000","last_peered":"2024-04-23T01:54:30.813603+0000","last_clean":"2024-04-23T01:54:30.813603+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:54:30.813603+0000","last_undegraded":"2024-04-23T01:54:30.813603+0000","last_fullsized":"2024-04-23T01:54:30.813603+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"57'152","reported_seq":178,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.644241+0000","last_change":"2024-04-23T01:54:15.668120+0000","last_active":"2024-04-23T01:54:31.644241+0000","last_peered":"2024-04-23T01:54:31.644241+0000","last_clean":"2024-04-23T01:54:31.644241+0000","last_became_active":"2024-04-23T01:54:15.661561+0000","last_became_peered":"2024-04-23T01:54:15.661561+0000","last_unstale":"2024-04-23T01:54:31.644241+0000","last_undegraded":"2024-04-23T01:54:31.644241+0000","last_fullsized":"2024-04-23T01:54:31.644241+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":132,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.619496+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:54:31.619496+0000","last_peered":"2024-04-23T01:54:31.619496+0000","last_clean":"2024-04-23T01:54:31.619496+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:54:31.619496+0000","last_undegraded":"2024-04-23T01:54:31.619496+0000","last_fullsized":"2024-04-23T01:54:31.619496+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"57'112","reported_seq":138,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.602437+0000","last_change":"2024-04-23T01:54:15.677434+0000","last_active":"2024-04-23T01:54:31.602437+0000","last_peered":"2024-04-23T01:54:31.602437+0000","last_clean":"2024-04-23T01:54:31.602437+0000","last_became_active":"2024-04-23T01:54:15.670406+0000","last_became_peered":"2024-04-23T01:54:15.670406+0000","last_unstale":"2024-04-23T01:54:31.602437+0000","last_undegraded":"2024-04-23T01:54:31.602437+0000","last_fullsized":"2024-04-23T01:54:31.602437+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":126,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.652887+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:54:31.652887+0000","last_peered":"2024-04-23T01:54:31.652887+0000","last_clean":"2024-04-23T01:54:31.652887+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:54:31.652887+0000","last_undegraded":"2024-04-23T01:54:31.652887+0000","last_fullsized":"2024-04-23T01:54:31.652887+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.e","version":"56'144","reported_seq":167,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:30.813776+0000","last_change":"2024-04-23T01:54:15.652153+0000","last_active":"2024-04-23T01:54:30.813776+0000","last_peered":"2024-04-23T01:54:30.813776+0000","last_clean":"2024-04-23T01:54:30.813776+0000","last_became_active":"2024-04-23T01:54:15.644332+0000","last_became_peered":"2024-04-23T01:54:15.644332+0000","last_unstale":"2024-04-23T01:54:30.813776+0000","last_undegraded":"2024-04-23T01:54:30.813776+0000","last_fullsized":"2024-04-23T01:54:30.813776+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":132,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.243172+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:54:31.243172+0000","last_peered":"2024-04-23T01:54:31.243172+0000","last_clean":"2024-04-23T01:54:31.243172+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:54:31.243172+0000","last_undegraded":"2024-04-23T01:54:31.243172+0000","last_fullsized":"2024-04-23T01:54:31.243172+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"57'152","reported_seq":178,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.611396+0000","last_change":"2024-04-23T01:54:15.637842+0000","last_active":"2024-04-23T01:54:31.611396+0000","last_peered":"2024-04-23T01:54:31.611396+0000","last_clean":"2024-04-23T01:54:31.611396+0000","last_became_active":"2024-04-23T01:54:15.631530+0000","last_became_peered":"2024-04-23T01:54:15.631530+0000","last_unstale":"2024-04-23T01:54:31.611396+0000","last_undegraded":"2024-04-23T01:54:31.611396+0000","last_fullsized":"2024-04-23T01:54:31.611396+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.617128+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:54:31.617128+0000","last_peered":"2024-04-23T01:54:31.617128+0000","last_clean":"2024-04-23T01:54:31.617128+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:54:31.617128+0000","last_undegraded":"2024-04-23T01:54:31.617128+0000","last_fullsized":"2024-04-23T01:54:31.617128+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"57'152","reported_seq":176,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.069621+0000","last_change":"2024-04-23T01:54:15.617410+0000","last_active":"2024-04-23T01:54:31.069621+0000","last_peered":"2024-04-23T01:54:31.069621+0000","last_clean":"2024-04-23T01:54:31.069621+0000","last_became_active":"2024-04-23T01:54:15.610208+0000","last_became_peered":"2024-04-23T01:54:15.610208+0000","last_unstale":"2024-04-23T01:54:31.069621+0000","last_undegraded":"2024-04-23T01:54:31.069621+0000","last_fullsized":"2024-04-23T01:54:31.069621+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":132,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.253606+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:54:31.253606+0000","last_peered":"2024-04-23T01:54:31.253606+0000","last_clean":"2024-04-23T01:54:31.253606+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:54:31.253606+0000","last_undegraded":"2024-04-23T01:54:31.253606+0000","last_fullsized":"2024-04-23T01:54:31.253606+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"56'88","reported_seq":114,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.645352+0000","last_change":"2024-04-23T01:54:15.630052+0000","last_active":"2024-04-23T01:54:31.645352+0000","last_peered":"2024-04-23T01:54:31.645352+0000","last_clean":"2024-04-23T01:54:31.645352+0000","last_became_active":"2024-04-23T01:54:15.624615+0000","last_became_peered":"2024-04-23T01:54:15.624615+0000","last_unstale":"2024-04-23T01:54:31.645352+0000","last_undegraded":"2024-04-23T01:54:31.645352+0000","last_fullsized":"2024-04-23T01:54:31.645352+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":720896,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":132,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.599009+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:54:31.599009+0000","last_peered":"2024-04-23T01:54:31.599009+0000","last_clean":"2024-04-23T01:54:31.599009+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:54:31.599009+0000","last_undegraded":"2024-04-23T01:54:31.599009+0000","last_fullsized":"2024-04-23T01:54:31.599009+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":18006016,"num_objects":276,"num_object_clones":0,"num_object_copies":828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2198,"num_write_kb":17584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":26980352,"data_stored":53960704,"data_compressed":2782971,"data_compressed_allocated":26980352,"data_compressed_original":53960704,"omap_allocated":0,"internal_metadata":0},"log_size":2194,"ondisk_log_size":2194,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706228,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":100824,"kb_used_data":7376,"kb_used_omap":0,"kb_used_meta":93440,"kb_avail":94271016,"statfs":{"total":96636764160,"available":96533520384,"internally_reserved":0,"allocated":7553024,"data_stored":14766600,"data_compressed":752010,"data_compressed_allocated":7041024,"data_compressed_original":14573568,"omap_allocated":0,"internal_metadata":95682560},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706228,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":98656,"kb_used_data":5208,"kb_used_omap":0,"kb_used_meta":93440,"kb_avail":94273184,"statfs":{"total":96636764160,"available":96535740416,"internally_reserved":0,"allocated":5332992,"data_stored":10313397,"data_compressed":522160,"data_compressed_allocated":4812800,"data_compressed_original":10117120,"omap_allocated":0,"internal_metadata":95682560},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706228,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":89092,"kb_used_data":8124,"kb_used_omap":0,"kb_used_meta":80960,"kb_avail":94282748,"statfs":{"total":96636764160,"available":96545533952,"internally_reserved":0,"allocated":8318976,"data_stored":15831016,"data_compressed":807339,"data_compressed_allocated":7827456,"data_compressed_original":15654912,"omap_allocated":0,"internal_metadata":82903040},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706228,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":100664,"kb_used_data":7216,"kb_used_omap":0,"kb_used_meta":93440,"kb_avail":94271176,"statfs":{"total":96636764160,"available":96533684224,"internally_reserved":0,"allocated":7389184,"data_stored":14438920,"data_compressed":735062,"data_compressed_allocated":6877184,"data_compressed_original":14245888,"omap_allocated":0,"internal_metadata":95682560},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6914048,"data_stored":13828096,"data_compressed":713146,"data_compressed_allocated":6914048,"data_compressed_original":13828096,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7942144,"data_stored":15884288,"data_compressed":819183,"data_compressed_allocated":7942144,"data_compressed_original":15884288,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4960256,"data_stored":9920512,"data_compressed":511665,"data_compressed_allocated":4960256,"data_compressed_original":9920512,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7163904,"data_stored":14327808,"data_compressed":738977,"data_compressed_allocated":7163904,"data_compressed_original":14327808,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:54:33.686 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:54:33.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:33.709+0000 7f794ff5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:33.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:33.717+0000 7f794ff5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:33.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:33.809+0000 7fc4482ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:33.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:33.820+0000 7fc4482ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:34.045 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:54:34.045 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:54:34.061 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2024-04-23T01:54:33.657446+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":18596384,"num_objects":278,"num_object_clones":0,"num_object_copies":834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":2335,"num_write_kb":19478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2291,"ondisk_log_size":2291,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":389236,"kb_used_data":27924,"kb_used_omap":0,"kb_used_meta":361280,"kb_avail":377098124,"statfs":{"total":386547056640,"available":386148478976,"internally_reserved":0,"allocated":28594176,"data_stored":55349933,"data_compressed":2816571,"data_compressed_allocated":26558464,"data_compressed_original":54591488,"omap_allocated":0,"internal_metadata":369950720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6070272,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":741,"num_write_kb":5928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000684"},"pg_stats":[{"pgid":"8.7","version":"58'183","reported_seq":210,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.480114+0000","last_change":"2024-04-23T01:54:15.581616+0000","last_active":"2024-04-23T01:54:31.480114+0000","last_peered":"2024-04-23T01:54:31.480114+0000","last_clean":"2024-04-23T01:54:31.480114+0000","last_became_active":"2024-04-23T01:54:15.576738+0000","last_became_peered":"2024-04-23T01:54:15.576738+0000","last_unstale":"2024-04-23T01:54:31.480114+0000","last_undegraded":"2024-04-23T01:54:31.480114+0000","last_fullsized":"2024-04-23T01:54:31.480114+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"58'111","reported_seq":138,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.699628+0000","last_change":"2024-04-23T01:54:15.596716+0000","last_active":"2024-04-23T01:54:31.699628+0000","last_peered":"2024-04-23T01:54:31.699628+0000","last_clean":"2024-04-23T01:54:31.699628+0000","last_became_active":"2024-04-23T01:54:15.589907+0000","last_became_peered":"2024-04-23T01:54:15.589907+0000","last_unstale":"2024-04-23T01:54:31.699628+0000","last_undegraded":"2024-04-23T01:54:31.699628+0000","last_fullsized":"2024-04-23T01:54:31.699628+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"58'97","reported_seq":124,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.617346+0000","last_change":"2024-04-23T01:54:15.700282+0000","last_active":"2024-04-23T01:54:31.617346+0000","last_peered":"2024-04-23T01:54:31.617346+0000","last_clean":"2024-04-23T01:54:31.617346+0000","last_became_active":"2024-04-23T01:54:15.686441+0000","last_became_peered":"2024-04-23T01:54:15.686441+0000","last_unstale":"2024-04-23T01:54:31.617346+0000","last_undegraded":"2024-04-23T01:54:31.617346+0000","last_fullsized":"2024-04-23T01:54:31.617346+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":802816,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98,"num_write_kb":784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"56'120","reported_seq":143,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:30.823497+0000","last_change":"2024-04-23T01:54:15.681870+0000","last_active":"2024-04-23T01:54:30.823497+0000","last_peered":"2024-04-23T01:54:30.823497+0000","last_clean":"2024-04-23T01:54:30.823497+0000","last_became_active":"2024-04-23T01:54:15.677678+0000","last_became_peered":"2024-04-23T01:54:15.677678+0000","last_unstale":"2024-04-23T01:54:30.823497+0000","last_undegraded":"2024-04-23T01:54:30.823497+0000","last_fullsized":"2024-04-23T01:54:30.823497+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3","version":"56'112","reported_seq":138,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.615576+0000","last_change":"2024-04-23T01:54:15.711951+0000","last_active":"2024-04-23T01:54:31.615576+0000","last_peered":"2024-04-23T01:54:31.615576+0000","last_clean":"2024-04-23T01:54:31.615576+0000","last_became_active":"2024-04-23T01:54:15.696611+0000","last_became_peered":"2024-04-23T01:54:15.696611+0000","last_unstale":"2024-04-23T01:54:31.615576+0000","last_undegraded":"2024-04-23T01:54:31.615576+0000","last_fullsized":"2024-04-23T01:54:31.615576+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"57'88","reported_seq":114,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.668306+0000","last_change":"2024-04-23T01:54:15.676654+0000","last_active":"2024-04-23T01:54:31.668306+0000","last_peered":"2024-04-23T01:54:31.668306+0000","last_clean":"2024-04-23T01:54:31.668306+0000","last_became_active":"2024-04-23T01:54:15.672239+0000","last_became_peered":"2024-04-23T01:54:15.672239+0000","last_unstale":"2024-04-23T01:54:31.668306+0000","last_undegraded":"2024-04-23T01:54:31.668306+0000","last_fullsized":"2024-04-23T01:54:31.668306+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":720896,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"58'172","reported_seq":199,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.670957+0000","last_change":"2024-04-23T01:54:15.609601+0000","last_active":"2024-04-23T01:54:31.670957+0000","last_peered":"2024-04-23T01:54:31.670957+0000","last_clean":"2024-04-23T01:54:31.670957+0000","last_became_active":"2024-04-23T01:54:15.598424+0000","last_became_peered":"2024-04-23T01:54:15.598424+0000","last_unstale":"2024-04-23T01:54:31.670957+0000","last_undegraded":"2024-04-23T01:54:31.670957+0000","last_fullsized":"2024-04-23T01:54:31.670957+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1417216,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":173,"num_write_kb":1384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"57'176","reported_seq":202,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.605634+0000","last_change":"2024-04-23T01:54:15.718344+0000","last_active":"2024-04-23T01:54:31.605634+0000","last_peered":"2024-04-23T01:54:31.605634+0000","last_clean":"2024-04-23T01:54:31.605634+0000","last_became_active":"2024-04-23T01:54:15.706071+0000","last_became_peered":"2024-04-23T01:54:15.706071+0000","last_unstale":"2024-04-23T01:54:31.605634+0000","last_undegraded":"2024-04-23T01:54:31.605634+0000","last_fullsized":"2024-04-23T01:54:31.605634+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"57'167","reported_seq":191,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.595726+0000","last_change":"2024-04-23T01:54:15.596696+0000","last_active":"2024-04-23T01:54:31.595726+0000","last_peered":"2024-04-23T01:54:31.595726+0000","last_clean":"2024-04-23T01:54:31.595726+0000","last_became_active":"2024-04-23T01:54:15.588060+0000","last_became_peered":"2024-04-23T01:54:15.588060+0000","last_unstale":"2024-04-23T01:54:31.595726+0000","last_undegraded":"2024-04-23T01:54:31.595726+0000","last_fullsized":"2024-04-23T01:54:31.595726+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1368064,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":167,"num_write_kb":1336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":261,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.616318+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:54:31.616318+0000","last_peered":"2024-04-23T01:54:31.616318+0000","last_clean":"2024-04-23T01:54:31.616318+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:54:31.616318+0000","last_undegraded":"2024-04-23T01:54:31.616318+0000","last_fullsized":"2024-04-23T01:54:31.616318+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":132,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.621120+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:54:31.621120+0000","last_peered":"2024-04-23T01:54:31.621120+0000","last_clean":"2024-04-23T01:54:31.621120+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:54:31.621120+0000","last_undegraded":"2024-04-23T01:54:31.621120+0000","last_fullsized":"2024-04-23T01:54:31.621120+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"57'168","reported_seq":194,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.633231+0000","last_change":"2024-04-23T01:54:15.705667+0000","last_active":"2024-04-23T01:54:31.633231+0000","last_peered":"2024-04-23T01:54:31.633231+0000","last_clean":"2024-04-23T01:54:31.633231+0000","last_became_active":"2024-04-23T01:54:15.701510+0000","last_became_peered":"2024-04-23T01:54:15.701510+0000","last_unstale":"2024-04-23T01:54:31.633231+0000","last_undegraded":"2024-04-23T01:54:31.633231+0000","last_fullsized":"2024-04-23T01:54:31.633231+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":1344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":123,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:30.813603+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:54:30.813603+0000","last_peered":"2024-04-23T01:54:30.813603+0000","last_clean":"2024-04-23T01:54:30.813603+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:54:30.813603+0000","last_undegraded":"2024-04-23T01:54:30.813603+0000","last_fullsized":"2024-04-23T01:54:30.813603+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"57'152","reported_seq":178,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.644241+0000","last_change":"2024-04-23T01:54:15.668120+0000","last_active":"2024-04-23T01:54:31.644241+0000","last_peered":"2024-04-23T01:54:31.644241+0000","last_clean":"2024-04-23T01:54:31.644241+0000","last_became_active":"2024-04-23T01:54:15.661561+0000","last_became_peered":"2024-04-23T01:54:15.661561+0000","last_unstale":"2024-04-23T01:54:31.644241+0000","last_undegraded":"2024-04-23T01:54:31.644241+0000","last_fullsized":"2024-04-23T01:54:31.644241+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":132,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.619496+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:54:31.619496+0000","last_peered":"2024-04-23T01:54:31.619496+0000","last_clean":"2024-04-23T01:54:31.619496+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:54:31.619496+0000","last_undegraded":"2024-04-23T01:54:31.619496+0000","last_fullsized":"2024-04-23T01:54:31.619496+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"57'112","reported_seq":138,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.602437+0000","last_change":"2024-04-23T01:54:15.677434+0000","last_active":"2024-04-23T01:54:31.602437+0000","last_peered":"2024-04-23T01:54:31.602437+0000","last_clean":"2024-04-23T01:54:31.602437+0000","last_became_active":"2024-04-23T01:54:15.670406+0000","last_became_peered":"2024-04-23T01:54:15.670406+0000","last_unstale":"2024-04-23T01:54:31.602437+0000","last_undegraded":"2024-04-23T01:54:31.602437+0000","last_fullsized":"2024-04-23T01:54:31.602437+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":126,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.652887+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:54:31.652887+0000","last_peered":"2024-04-23T01:54:31.652887+0000","last_clean":"2024-04-23T01:54:31.652887+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:54:31.652887+0000","last_undegraded":"2024-04-23T01:54:31.652887+0000","last_fullsized":"2024-04-23T01:54:31.652887+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.e","version":"56'144","reported_seq":167,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:30.813776+0000","last_change":"2024-04-23T01:54:15.652153+0000","last_active":"2024-04-23T01:54:30.813776+0000","last_peered":"2024-04-23T01:54:30.813776+0000","last_clean":"2024-04-23T01:54:30.813776+0000","last_became_active":"2024-04-23T01:54:15.644332+0000","last_became_peered":"2024-04-23T01:54:15.644332+0000","last_unstale":"2024-04-23T01:54:30.813776+0000","last_undegraded":"2024-04-23T01:54:30.813776+0000","last_fullsized":"2024-04-23T01:54:30.813776+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":132,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.243172+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:54:31.243172+0000","last_peered":"2024-04-23T01:54:31.243172+0000","last_clean":"2024-04-23T01:54:31.243172+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:54:31.243172+0000","last_undegraded":"2024-04-23T01:54:31.243172+0000","last_fullsized":"2024-04-23T01:54:31.243172+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"57'152","reported_seq":178,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.611396+0000","last_change":"2024-04-23T01:54:15.637842+0000","last_active":"2024-04-23T01:54:31.611396+0000","last_peered":"2024-04-23T01:54:31.611396+0000","last_clean":"2024-04-23T01:54:31.611396+0000","last_became_active":"2024-04-23T01:54:15.631530+0000","last_became_peered":"2024-04-23T01:54:15.631530+0000","last_unstale":"2024-04-23T01:54:31.611396+0000","last_undegraded":"2024-04-23T01:54:31.611396+0000","last_fullsized":"2024-04-23T01:54:31.611396+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.617128+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:54:31.617128+0000","last_peered":"2024-04-23T01:54:31.617128+0000","last_clean":"2024-04-23T01:54:31.617128+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:54:31.617128+0000","last_undegraded":"2024-04-23T01:54:31.617128+0000","last_fullsized":"2024-04-23T01:54:31.617128+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"57'152","reported_seq":176,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.069621+0000","last_change":"2024-04-23T01:54:15.617410+0000","last_active":"2024-04-23T01:54:31.069621+0000","last_peered":"2024-04-23T01:54:31.069621+0000","last_clean":"2024-04-23T01:54:31.069621+0000","last_became_active":"2024-04-23T01:54:15.610208+0000","last_became_peered":"2024-04-23T01:54:15.610208+0000","last_unstale":"2024-04-23T01:54:31.069621+0000","last_undegraded":"2024-04-23T01:54:31.069621+0000","last_fullsized":"2024-04-23T01:54:31.069621+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":132,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.253606+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:54:31.253606+0000","last_peered":"2024-04-23T01:54:31.253606+0000","last_clean":"2024-04-23T01:54:31.253606+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:54:31.253606+0000","last_undegraded":"2024-04-23T01:54:31.253606+0000","last_fullsized":"2024-04-23T01:54:31.253606+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"56'88","reported_seq":114,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.645352+0000","last_change":"2024-04-23T01:54:15.630052+0000","last_active":"2024-04-23T01:54:31.645352+0000","last_peered":"2024-04-23T01:54:31.645352+0000","last_clean":"2024-04-23T01:54:31.645352+0000","last_became_active":"2024-04-23T01:54:15.624615+0000","last_became_peered":"2024-04-23T01:54:15.624615+0000","last_unstale":"2024-04-23T01:54:31.645352+0000","last_undegraded":"2024-04-23T01:54:31.645352+0000","last_fullsized":"2024-04-23T01:54:31.645352+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":720896,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":132,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.599009+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:54:31.599009+0000","last_peered":"2024-04-23T01:54:31.599009+0000","last_clean":"2024-04-23T01:54:31.599009+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:54:31.599009+0000","last_undegraded":"2024-04-23T01:54:31.599009+0000","last_fullsized":"2024-04-23T01:54:31.599009+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":18006016,"num_objects":276,"num_object_clones":0,"num_object_copies":828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2198,"num_write_kb":17584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":26980352,"data_stored":53960704,"data_compressed":2782971,"data_compressed_allocated":26980352,"data_compressed_original":53960704,"omap_allocated":0,"internal_metadata":0},"log_size":2194,"ondisk_log_size":2194,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706228,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":100824,"kb_used_data":7376,"kb_used_omap":0,"kb_used_meta":93440,"kb_avail":94271016,"statfs":{"total":96636764160,"available":96533520384,"internally_reserved":0,"allocated":7553024,"data_stored":14766600,"data_compressed":752010,"data_compressed_allocated":7041024,"data_compressed_original":14573568,"omap_allocated":0,"internal_metadata":95682560},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706228,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":98656,"kb_used_data":5208,"kb_used_omap":0,"kb_used_meta":93440,"kb_avail":94273184,"statfs":{"total":96636764160,"available":96535740416,"internally_reserved":0,"allocated":5332992,"data_stored":10313397,"data_compressed":522160,"data_compressed_allocated":4812800,"data_compressed_original":10117120,"omap_allocated":0,"internal_metadata":95682560},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706228,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":89092,"kb_used_data":8124,"kb_used_omap":0,"kb_used_meta":80960,"kb_avail":94282748,"statfs":{"total":96636764160,"available":96545533952,"internally_reserved":0,"allocated":8318976,"data_stored":15831016,"data_compressed":807339,"data_compressed_allocated":7827456,"data_compressed_original":15654912,"omap_allocated":0,"internal_metadata":82903040},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706228,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":100664,"kb_used_data":7216,"kb_used_omap":0,"kb_used_meta":93440,"kb_avail":94271176,"statfs":{"total":96636764160,"available":96533684224,"internally_reserved":0,"allocated":7389184,"data_stored":14438920,"data_compressed":735062,"data_compressed_allocated":6877184,"data_compressed_original":14245888,"omap_allocated":0,"internal_metadata":95682560},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6914048,"data_stored":13828096,"data_compressed":713146,"data_compressed_allocated":6914048,"data_compressed_original":13828096,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7942144,"data_stored":15884288,"data_compressed":819183,"data_compressed_allocated":7942144,"data_compressed_original":15884288,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4960256,"data_stored":9920512,"data_compressed":511665,"data_compressed_allocated":4960256,"data_compressed_original":9920512,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7163904,"data_stored":14327808,"data_compressed":738977,"data_compressed_allocated":7163904,"data_compressed_original":14327808,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:54:34.062 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-23T01:54:34.063 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:54:34.063 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:54:34.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:34.183+0000 7fdf7415d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:34.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:34.194+0000 7fdf7415d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:34.432 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:54:34.432 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:54:34.451 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2024-04-23T01:54:33.657446+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":18596384,"num_objects":278,"num_object_clones":0,"num_object_copies":834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":2335,"num_write_kb":19478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2291,"ondisk_log_size":2291,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":389236,"kb_used_data":27924,"kb_used_omap":0,"kb_used_meta":361280,"kb_avail":377098124,"statfs":{"total":386547056640,"available":386148478976,"internally_reserved":0,"allocated":28594176,"data_stored":55349933,"data_compressed":2816571,"data_compressed_allocated":26558464,"data_compressed_original":54591488,"omap_allocated":0,"internal_metadata":369950720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6070272,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":741,"num_write_kb":5928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000684"},"pg_stats":[{"pgid":"8.7","version":"58'183","reported_seq":210,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.480114+0000","last_change":"2024-04-23T01:54:15.581616+0000","last_active":"2024-04-23T01:54:31.480114+0000","last_peered":"2024-04-23T01:54:31.480114+0000","last_clean":"2024-04-23T01:54:31.480114+0000","last_became_active":"2024-04-23T01:54:15.576738+0000","last_became_peered":"2024-04-23T01:54:15.576738+0000","last_unstale":"2024-04-23T01:54:31.480114+0000","last_undegraded":"2024-04-23T01:54:31.480114+0000","last_fullsized":"2024-04-23T01:54:31.480114+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"58'111","reported_seq":138,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.699628+0000","last_change":"2024-04-23T01:54:15.596716+0000","last_active":"2024-04-23T01:54:31.699628+0000","last_peered":"2024-04-23T01:54:31.699628+0000","last_clean":"2024-04-23T01:54:31.699628+0000","last_became_active":"2024-04-23T01:54:15.589907+0000","last_became_peered":"2024-04-23T01:54:15.589907+0000","last_unstale":"2024-04-23T01:54:31.699628+0000","last_undegraded":"2024-04-23T01:54:31.699628+0000","last_fullsized":"2024-04-23T01:54:31.699628+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"58'97","reported_seq":124,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.617346+0000","last_change":"2024-04-23T01:54:15.700282+0000","last_active":"2024-04-23T01:54:31.617346+0000","last_peered":"2024-04-23T01:54:31.617346+0000","last_clean":"2024-04-23T01:54:31.617346+0000","last_became_active":"2024-04-23T01:54:15.686441+0000","last_became_peered":"2024-04-23T01:54:15.686441+0000","last_unstale":"2024-04-23T01:54:31.617346+0000","last_undegraded":"2024-04-23T01:54:31.617346+0000","last_fullsized":"2024-04-23T01:54:31.617346+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":802816,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98,"num_write_kb":784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"56'120","reported_seq":143,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:30.823497+0000","last_change":"2024-04-23T01:54:15.681870+0000","last_active":"2024-04-23T01:54:30.823497+0000","last_peered":"2024-04-23T01:54:30.823497+0000","last_clean":"2024-04-23T01:54:30.823497+0000","last_became_active":"2024-04-23T01:54:15.677678+0000","last_became_peered":"2024-04-23T01:54:15.677678+0000","last_unstale":"2024-04-23T01:54:30.823497+0000","last_undegraded":"2024-04-23T01:54:30.823497+0000","last_fullsized":"2024-04-23T01:54:30.823497+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3","version":"56'112","reported_seq":138,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.615576+0000","last_change":"2024-04-23T01:54:15.711951+0000","last_active":"2024-04-23T01:54:31.615576+0000","last_peered":"2024-04-23T01:54:31.615576+0000","last_clean":"2024-04-23T01:54:31.615576+0000","last_became_active":"2024-04-23T01:54:15.696611+0000","last_became_peered":"2024-04-23T01:54:15.696611+0000","last_unstale":"2024-04-23T01:54:31.615576+0000","last_undegraded":"2024-04-23T01:54:31.615576+0000","last_fullsized":"2024-04-23T01:54:31.615576+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"57'88","reported_seq":114,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.668306+0000","last_change":"2024-04-23T01:54:15.676654+0000","last_active":"2024-04-23T01:54:31.668306+0000","last_peered":"2024-04-23T01:54:31.668306+0000","last_clean":"2024-04-23T01:54:31.668306+0000","last_became_active":"2024-04-23T01:54:15.672239+0000","last_became_peered":"2024-04-23T01:54:15.672239+0000","last_unstale":"2024-04-23T01:54:31.668306+0000","last_undegraded":"2024-04-23T01:54:31.668306+0000","last_fullsized":"2024-04-23T01:54:31.668306+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":720896,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"58'172","reported_seq":199,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.670957+0000","last_change":"2024-04-23T01:54:15.609601+0000","last_active":"2024-04-23T01:54:31.670957+0000","last_peered":"2024-04-23T01:54:31.670957+0000","last_clean":"2024-04-23T01:54:31.670957+0000","last_became_active":"2024-04-23T01:54:15.598424+0000","last_became_peered":"2024-04-23T01:54:15.598424+0000","last_unstale":"2024-04-23T01:54:31.670957+0000","last_undegraded":"2024-04-23T01:54:31.670957+0000","last_fullsized":"2024-04-23T01:54:31.670957+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1417216,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":173,"num_write_kb":1384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"57'176","reported_seq":202,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.605634+0000","last_change":"2024-04-23T01:54:15.718344+0000","last_active":"2024-04-23T01:54:31.605634+0000","last_peered":"2024-04-23T01:54:31.605634+0000","last_clean":"2024-04-23T01:54:31.605634+0000","last_became_active":"2024-04-23T01:54:15.706071+0000","last_became_peered":"2024-04-23T01:54:15.706071+0000","last_unstale":"2024-04-23T01:54:31.605634+0000","last_undegraded":"2024-04-23T01:54:31.605634+0000","last_fullsized":"2024-04-23T01:54:31.605634+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"57'167","reported_seq":191,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.595726+0000","last_change":"2024-04-23T01:54:15.596696+0000","last_active":"2024-04-23T01:54:31.595726+0000","last_peered":"2024-04-23T01:54:31.595726+0000","last_clean":"2024-04-23T01:54:31.595726+0000","last_became_active":"2024-04-23T01:54:15.588060+0000","last_became_peered":"2024-04-23T01:54:15.588060+0000","last_unstale":"2024-04-23T01:54:31.595726+0000","last_undegraded":"2024-04-23T01:54:31.595726+0000","last_fullsized":"2024-04-23T01:54:31.595726+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1368064,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":167,"num_write_kb":1336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":261,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.616318+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:54:31.616318+0000","last_peered":"2024-04-23T01:54:31.616318+0000","last_clean":"2024-04-23T01:54:31.616318+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:54:31.616318+0000","last_undegraded":"2024-04-23T01:54:31.616318+0000","last_fullsized":"2024-04-23T01:54:31.616318+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":132,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.621120+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:54:31.621120+0000","last_peered":"2024-04-23T01:54:31.621120+0000","last_clean":"2024-04-23T01:54:31.621120+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:54:31.621120+0000","last_undegraded":"2024-04-23T01:54:31.621120+0000","last_fullsized":"2024-04-23T01:54:31.621120+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"57'168","reported_seq":194,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.633231+0000","last_change":"2024-04-23T01:54:15.705667+0000","last_active":"2024-04-23T01:54:31.633231+0000","last_peered":"2024-04-23T01:54:31.633231+0000","last_clean":"2024-04-23T01:54:31.633231+0000","last_became_active":"2024-04-23T01:54:15.701510+0000","last_became_peered":"2024-04-23T01:54:15.701510+0000","last_unstale":"2024-04-23T01:54:31.633231+0000","last_undegraded":"2024-04-23T01:54:31.633231+0000","last_fullsized":"2024-04-23T01:54:31.633231+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":1344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":123,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:30.813603+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:54:30.813603+0000","last_peered":"2024-04-23T01:54:30.813603+0000","last_clean":"2024-04-23T01:54:30.813603+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:54:30.813603+0000","last_undegraded":"2024-04-23T01:54:30.813603+0000","last_fullsized":"2024-04-23T01:54:30.813603+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"57'152","reported_seq":178,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.644241+0000","last_change":"2024-04-23T01:54:15.668120+0000","last_active":"2024-04-23T01:54:31.644241+0000","last_peered":"2024-04-23T01:54:31.644241+0000","last_clean":"2024-04-23T01:54:31.644241+0000","last_became_active":"2024-04-23T01:54:15.661561+0000","last_became_peered":"2024-04-23T01:54:15.661561+0000","last_unstale":"2024-04-23T01:54:31.644241+0000","last_undegraded":"2024-04-23T01:54:31.644241+0000","last_fullsized":"2024-04-23T01:54:31.644241+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":132,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.619496+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:54:31.619496+0000","last_peered":"2024-04-23T01:54:31.619496+0000","last_clean":"2024-04-23T01:54:31.619496+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:54:31.619496+0000","last_undegraded":"2024-04-23T01:54:31.619496+0000","last_fullsized":"2024-04-23T01:54:31.619496+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"57'112","reported_seq":138,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.602437+0000","last_change":"2024-04-23T01:54:15.677434+0000","last_active":"2024-04-23T01:54:31.602437+0000","last_peered":"2024-04-23T01:54:31.602437+0000","last_clean":"2024-04-23T01:54:31.602437+0000","last_became_active":"2024-04-23T01:54:15.670406+0000","last_became_peered":"2024-04-23T01:54:15.670406+0000","last_unstale":"2024-04-23T01:54:31.602437+0000","last_undegraded":"2024-04-23T01:54:31.602437+0000","last_fullsized":"2024-04-23T01:54:31.602437+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":126,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.652887+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:54:31.652887+0000","last_peered":"2024-04-23T01:54:31.652887+0000","last_clean":"2024-04-23T01:54:31.652887+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:54:31.652887+0000","last_undegraded":"2024-04-23T01:54:31.652887+0000","last_fullsized":"2024-04-23T01:54:31.652887+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.e","version":"56'144","reported_seq":167,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:30.813776+0000","last_change":"2024-04-23T01:54:15.652153+0000","last_active":"2024-04-23T01:54:30.813776+0000","last_peered":"2024-04-23T01:54:30.813776+0000","last_clean":"2024-04-23T01:54:30.813776+0000","last_became_active":"2024-04-23T01:54:15.644332+0000","last_became_peered":"2024-04-23T01:54:15.644332+0000","last_unstale":"2024-04-23T01:54:30.813776+0000","last_undegraded":"2024-04-23T01:54:30.813776+0000","last_fullsized":"2024-04-23T01:54:30.813776+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":132,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.243172+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:54:31.243172+0000","last_peered":"2024-04-23T01:54:31.243172+0000","last_clean":"2024-04-23T01:54:31.243172+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:54:31.243172+0000","last_undegraded":"2024-04-23T01:54:31.243172+0000","last_fullsized":"2024-04-23T01:54:31.243172+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"57'152","reported_seq":178,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.611396+0000","last_change":"2024-04-23T01:54:15.637842+0000","last_active":"2024-04-23T01:54:31.611396+0000","last_peered":"2024-04-23T01:54:31.611396+0000","last_clean":"2024-04-23T01:54:31.611396+0000","last_became_active":"2024-04-23T01:54:15.631530+0000","last_became_peered":"2024-04-23T01:54:15.631530+0000","last_unstale":"2024-04-23T01:54:31.611396+0000","last_undegraded":"2024-04-23T01:54:31.611396+0000","last_fullsized":"2024-04-23T01:54:31.611396+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.617128+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:54:31.617128+0000","last_peered":"2024-04-23T01:54:31.617128+0000","last_clean":"2024-04-23T01:54:31.617128+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:54:31.617128+0000","last_undegraded":"2024-04-23T01:54:31.617128+0000","last_fullsized":"2024-04-23T01:54:31.617128+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"57'152","reported_seq":176,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.069621+0000","last_change":"2024-04-23T01:54:15.617410+0000","last_active":"2024-04-23T01:54:31.069621+0000","last_peered":"2024-04-23T01:54:31.069621+0000","last_clean":"2024-04-23T01:54:31.069621+0000","last_became_active":"2024-04-23T01:54:15.610208+0000","last_became_peered":"2024-04-23T01:54:15.610208+0000","last_unstale":"2024-04-23T01:54:31.069621+0000","last_undegraded":"2024-04-23T01:54:31.069621+0000","last_fullsized":"2024-04-23T01:54:31.069621+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":132,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.253606+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:54:31.253606+0000","last_peered":"2024-04-23T01:54:31.253606+0000","last_clean":"2024-04-23T01:54:31.253606+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:54:31.253606+0000","last_undegraded":"2024-04-23T01:54:31.253606+0000","last_fullsized":"2024-04-23T01:54:31.253606+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"56'88","reported_seq":114,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.645352+0000","last_change":"2024-04-23T01:54:15.630052+0000","last_active":"2024-04-23T01:54:31.645352+0000","last_peered":"2024-04-23T01:54:31.645352+0000","last_clean":"2024-04-23T01:54:31.645352+0000","last_became_active":"2024-04-23T01:54:15.624615+0000","last_became_peered":"2024-04-23T01:54:15.624615+0000","last_unstale":"2024-04-23T01:54:31.645352+0000","last_undegraded":"2024-04-23T01:54:31.645352+0000","last_fullsized":"2024-04-23T01:54:31.645352+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":720896,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":132,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-23T01:54:31.599009+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:54:31.599009+0000","last_peered":"2024-04-23T01:54:31.599009+0000","last_clean":"2024-04-23T01:54:31.599009+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:54:31.599009+0000","last_undegraded":"2024-04-23T01:54:31.599009+0000","last_fullsized":"2024-04-23T01:54:31.599009+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":18006016,"num_objects":276,"num_object_clones":0,"num_object_copies":828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2198,"num_write_kb":17584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":26980352,"data_stored":53960704,"data_compressed":2782971,"data_compressed_allocated":26980352,"data_compressed_original":53960704,"omap_allocated":0,"internal_metadata":0},"log_size":2194,"ondisk_log_size":2194,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706228,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":100824,"kb_used_data":7376,"kb_used_omap":0,"kb_used_meta":93440,"kb_avail":94271016,"statfs":{"total":96636764160,"available":96533520384,"internally_reserved":0,"allocated":7553024,"data_stored":14766600,"data_compressed":752010,"data_compressed_allocated":7041024,"data_compressed_original":14573568,"omap_allocated":0,"internal_metadata":95682560},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706228,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":98656,"kb_used_data":5208,"kb_used_omap":0,"kb_used_meta":93440,"kb_avail":94273184,"statfs":{"total":96636764160,"available":96535740416,"internally_reserved":0,"allocated":5332992,"data_stored":10313397,"data_compressed":522160,"data_compressed_allocated":4812800,"data_compressed_original":10117120,"omap_allocated":0,"internal_metadata":95682560},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706228,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":89092,"kb_used_data":8124,"kb_used_omap":0,"kb_used_meta":80960,"kb_avail":94282748,"statfs":{"total":96636764160,"available":96545533952,"internally_reserved":0,"allocated":8318976,"data_stored":15831016,"data_compressed":807339,"data_compressed_allocated":7827456,"data_compressed_original":15654912,"omap_allocated":0,"internal_metadata":82903040},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706228,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":100664,"kb_used_data":7216,"kb_used_omap":0,"kb_used_meta":93440,"kb_avail":94271176,"statfs":{"total":96636764160,"available":96533684224,"internally_reserved":0,"allocated":7389184,"data_stored":14438920,"data_compressed":735062,"data_compressed_allocated":6877184,"data_compressed_original":14245888,"omap_allocated":0,"internal_metadata":95682560},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6914048,"data_stored":13828096,"data_compressed":713146,"data_compressed_allocated":6914048,"data_compressed_original":13828096,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7942144,"data_stored":15884288,"data_compressed":819183,"data_compressed_allocated":7942144,"data_compressed_original":15884288,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4960256,"data_stored":9920512,"data_compressed":511665,"data_compressed_allocated":4960256,"data_compressed_original":9920512,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7163904,"data_stored":14327808,"data_compressed":738977,"data_compressed_allocated":7163904,"data_compressed_original":14327808,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:54:34.520 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 18 16 2603 2587 1.12273 1.11719 0.0388627 0.110981 2024-04-23T01:54:35.521 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 19 16 2750 2734 1.12408 1.14844 0.168192 0.110832 2024-04-23T01:54:35.903 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:54:36.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:36.033+0000 7fdd648ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:36.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:36.041+0000 7fdd648ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:36.520 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:54:36.520271+0000 min lat: 0.0165503 max lat: 0.347473 avg lat: 0.110884 2024-04-23T01:54:36.520 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:54:36.520 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 20 16 2894 2878 1.12412 1.125 0.0657437 0.110884 2024-04-23T01:54:37.520 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 21 16 3042 3026 1.12565 1.15625 0.083303 0.110679 2024-04-23T01:54:38.209 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:54:38.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:38.336+0000 7f825d02d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:38.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:38.343+0000 7f825d02d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:38.520 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 22 16 3171 3155 1.12029 1.00781 0.228518 0.111228 2024-04-23T01:54:39.453 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:54:39.454 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:54:39.454 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:54:39.521 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 23 16 3331 3315 1.12592 1.25 0.0959314 0.11067 2024-04-23T01:54:39.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:39.581+0000 7f9e12ced640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:39.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:39.591+0000 7f9e12ced640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:39.946 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:54:39.946 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:54:39.965 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2024-04-23T01:54:39.658519+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":24420896,"num_objects":367,"num_object_clones":0,"num_object_copies":1101,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":3046,"num_write_kb":25166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2992,"ondisk_log_size":2992,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":401924,"kb_used_data":36452,"kb_used_omap":0,"kb_used_meta":365440,"kb_avail":377085436,"statfs":{"total":386547056640,"available":386135486464,"internally_reserved":0,"allocated":37326848,"data_stored":72670748,"data_compressed":3709101,"data_compressed_allocated":35201024,"data_compressed_original":71876608,"omap_allocated":0,"internal_metadata":374210560},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11894784,"num_objects":181,"num_object_clones":0,"num_object_copies":543,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1452,"num_write_kb":11616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001381"},"pg_stats":[{"pgid":"8.7","version":"60'199","reported_seq":230,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:36.082761+0000","last_change":"2024-04-23T01:54:15.581616+0000","last_active":"2024-04-23T01:54:36.082761+0000","last_peered":"2024-04-23T01:54:36.082761+0000","last_clean":"2024-04-23T01:54:36.082761+0000","last_became_active":"2024-04-23T01:54:15.576738+0000","last_became_peered":"2024-04-23T01:54:15.576738+0000","last_unstale":"2024-04-23T01:54:36.082761+0000","last_undegraded":"2024-04-23T01:54:36.082761+0000","last_fullsized":"2024-04-23T01:54:36.082761+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":200,"num_write_kb":1600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"60'167","reported_seq":198,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:36.633286+0000","last_change":"2024-04-23T01:54:15.596716+0000","last_active":"2024-04-23T01:54:36.633286+0000","last_peered":"2024-04-23T01:54:36.633286+0000","last_clean":"2024-04-23T01:54:36.633286+0000","last_became_active":"2024-04-23T01:54:15.589907+0000","last_became_peered":"2024-04-23T01:54:15.589907+0000","last_unstale":"2024-04-23T01:54:36.633286+0000","last_undegraded":"2024-04-23T01:54:36.633286+0000","last_fullsized":"2024-04-23T01:54:36.633286+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":1344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"60'148","reported_seq":181,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:36.628308+0000","last_change":"2024-04-23T01:54:15.700282+0000","last_active":"2024-04-23T01:54:36.628308+0000","last_peered":"2024-04-23T01:54:36.628308+0000","last_clean":"2024-04-23T01:54:36.628308+0000","last_became_active":"2024-04-23T01:54:15.686441+0000","last_became_peered":"2024-04-23T01:54:15.686441+0000","last_unstale":"2024-04-23T01:54:36.628308+0000","last_undegraded":"2024-04-23T01:54:36.628308+0000","last_fullsized":"2024-04-23T01:54:36.628308+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1220608,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":149,"num_write_kb":1192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"60'151","reported_seq":182,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.786822+0000","last_change":"2024-04-23T01:54:15.681870+0000","last_active":"2024-04-23T01:54:35.786822+0000","last_peered":"2024-04-23T01:54:35.786822+0000","last_clean":"2024-04-23T01:54:35.786822+0000","last_became_active":"2024-04-23T01:54:15.677678+0000","last_became_peered":"2024-04-23T01:54:15.677678+0000","last_unstale":"2024-04-23T01:54:35.786822+0000","last_undegraded":"2024-04-23T01:54:35.786822+0000","last_fullsized":"2024-04-23T01:54:35.786822+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3","version":"60'151","reported_seq":184,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:36.297872+0000","last_change":"2024-04-23T01:54:15.711951+0000","last_active":"2024-04-23T01:54:36.297872+0000","last_peered":"2024-04-23T01:54:36.297872+0000","last_clean":"2024-04-23T01:54:36.297872+0000","last_became_active":"2024-04-23T01:54:15.696611+0000","last_became_peered":"2024-04-23T01:54:15.696611+0000","last_unstale":"2024-04-23T01:54:36.297872+0000","last_undegraded":"2024-04-23T01:54:36.297872+0000","last_fullsized":"2024-04-23T01:54:36.297872+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"58'120","reported_seq":151,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.676149+0000","last_change":"2024-04-23T01:54:15.676654+0000","last_active":"2024-04-23T01:54:35.676149+0000","last_peered":"2024-04-23T01:54:35.676149+0000","last_clean":"2024-04-23T01:54:35.676149+0000","last_became_active":"2024-04-23T01:54:15.672239+0000","last_became_peered":"2024-04-23T01:54:15.672239+0000","last_unstale":"2024-04-23T01:54:35.676149+0000","last_undegraded":"2024-04-23T01:54:35.676149+0000","last_fullsized":"2024-04-23T01:54:35.676149+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":991232,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121,"num_write_kb":968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"60'238","reported_seq":271,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:36.695520+0000","last_change":"2024-04-23T01:54:15.609601+0000","last_active":"2024-04-23T01:54:36.695520+0000","last_peered":"2024-04-23T01:54:36.695520+0000","last_clean":"2024-04-23T01:54:36.695520+0000","last_became_active":"2024-04-23T01:54:15.598424+0000","last_became_peered":"2024-04-23T01:54:15.598424+0000","last_unstale":"2024-04-23T01:54:36.695520+0000","last_undegraded":"2024-04-23T01:54:36.695520+0000","last_fullsized":"2024-04-23T01:54:36.695520+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1957888,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":239,"num_write_kb":1912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"59'208","reported_seq":240,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.656517+0000","last_change":"2024-04-23T01:54:15.718344+0000","last_active":"2024-04-23T01:54:35.656517+0000","last_peered":"2024-04-23T01:54:35.656517+0000","last_clean":"2024-04-23T01:54:35.656517+0000","last_became_active":"2024-04-23T01:54:15.706071+0000","last_became_peered":"2024-04-23T01:54:15.706071+0000","last_unstale":"2024-04-23T01:54:35.656517+0000","last_undegraded":"2024-04-23T01:54:35.656517+0000","last_fullsized":"2024-04-23T01:54:35.656517+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703936,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":208,"num_write_kb":1664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"58'176","reported_seq":206,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.654493+0000","last_change":"2024-04-23T01:54:15.596696+0000","last_active":"2024-04-23T01:54:35.654493+0000","last_peered":"2024-04-23T01:54:35.654493+0000","last_clean":"2024-04-23T01:54:35.654493+0000","last_became_active":"2024-04-23T01:54:15.588060+0000","last_became_peered":"2024-04-23T01:54:15.588060+0000","last_unstale":"2024-04-23T01:54:35.654493+0000","last_undegraded":"2024-04-23T01:54:35.654493+0000","last_fullsized":"2024-04-23T01:54:35.654493+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":267,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.696720+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:54:35.696720+0000","last_peered":"2024-04-23T01:54:35.696720+0000","last_clean":"2024-04-23T01:54:35.696720+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:54:35.696720+0000","last_undegraded":"2024-04-23T01:54:35.696720+0000","last_fullsized":"2024-04-23T01:54:35.696720+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":138,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.679304+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:54:35.679304+0000","last_peered":"2024-04-23T01:54:35.679304+0000","last_clean":"2024-04-23T01:54:35.679304+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:54:35.679304+0000","last_undegraded":"2024-04-23T01:54:35.679304+0000","last_fullsized":"2024-04-23T01:54:35.679304+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"60'231","reported_seq":264,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:36.541464+0000","last_change":"2024-04-23T01:54:15.705667+0000","last_active":"2024-04-23T01:54:36.541464+0000","last_peered":"2024-04-23T01:54:36.541464+0000","last_clean":"2024-04-23T01:54:36.541464+0000","last_became_active":"2024-04-23T01:54:15.701510+0000","last_became_peered":"2024-04-23T01:54:15.701510+0000","last_unstale":"2024-04-23T01:54:36.541464+0000","last_undegraded":"2024-04-23T01:54:36.541464+0000","last_fullsized":"2024-04-23T01:54:36.541464+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1900544,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":232,"num_write_kb":1856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":130,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.677386+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:54:35.677386+0000","last_peered":"2024-04-23T01:54:35.677386+0000","last_clean":"2024-04-23T01:54:35.677386+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:54:35.677386+0000","last_undegraded":"2024-04-23T01:54:35.677386+0000","last_fullsized":"2024-04-23T01:54:35.677386+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"60'175","reported_seq":206,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:36.021795+0000","last_change":"2024-04-23T01:54:15.668120+0000","last_active":"2024-04-23T01:54:36.021795+0000","last_peered":"2024-04-23T01:54:36.021795+0000","last_clean":"2024-04-23T01:54:36.021795+0000","last_became_active":"2024-04-23T01:54:15.661561+0000","last_became_peered":"2024-04-23T01:54:15.661561+0000","last_unstale":"2024-04-23T01:54:36.021795+0000","last_undegraded":"2024-04-23T01:54:36.021795+0000","last_fullsized":"2024-04-23T01:54:36.021795+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":138,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.657300+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:54:35.657300+0000","last_peered":"2024-04-23T01:54:35.657300+0000","last_clean":"2024-04-23T01:54:35.657300+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:54:35.657300+0000","last_undegraded":"2024-04-23T01:54:35.657300+0000","last_fullsized":"2024-04-23T01:54:35.657300+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"60'159","reported_seq":192,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:36.191666+0000","last_change":"2024-04-23T01:54:15.677434+0000","last_active":"2024-04-23T01:54:36.191666+0000","last_peered":"2024-04-23T01:54:36.191666+0000","last_clean":"2024-04-23T01:54:36.191666+0000","last_became_active":"2024-04-23T01:54:15.670406+0000","last_became_peered":"2024-04-23T01:54:15.670406+0000","last_unstale":"2024-04-23T01:54:36.191666+0000","last_undegraded":"2024-04-23T01:54:36.191666+0000","last_fullsized":"2024-04-23T01:54:36.191666+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310720,"num_objects":20,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":160,"num_write_kb":1280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":130,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.679268+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:54:35.679268+0000","last_peered":"2024-04-23T01:54:35.679268+0000","last_clean":"2024-04-23T01:54:35.679268+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:54:35.679268+0000","last_undegraded":"2024-04-23T01:54:35.679268+0000","last_fullsized":"2024-04-23T01:54:35.679268+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.e","version":"60'191","reported_seq":222,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.878914+0000","last_change":"2024-04-23T01:54:15.652153+0000","last_active":"2024-04-23T01:54:35.878914+0000","last_peered":"2024-04-23T01:54:35.878914+0000","last_clean":"2024-04-23T01:54:35.878914+0000","last_became_active":"2024-04-23T01:54:15.644332+0000","last_became_peered":"2024-04-23T01:54:15.644332+0000","last_unstale":"2024-04-23T01:54:35.878914+0000","last_undegraded":"2024-04-23T01:54:35.878914+0000","last_fullsized":"2024-04-23T01:54:35.878914+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1572864,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":192,"num_write_kb":1536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":136,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:33.277036+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:54:33.277036+0000","last_peered":"2024-04-23T01:54:33.277036+0000","last_clean":"2024-04-23T01:54:33.277036+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:54:33.277036+0000","last_undegraded":"2024-04-23T01:54:33.277036+0000","last_fullsized":"2024-04-23T01:54:33.277036+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"60'247","reported_seq":280,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:36.433649+0000","last_change":"2024-04-23T01:54:15.637842+0000","last_active":"2024-04-23T01:54:36.433649+0000","last_peered":"2024-04-23T01:54:36.433649+0000","last_clean":"2024-04-23T01:54:36.433649+0000","last_became_active":"2024-04-23T01:54:15.631530+0000","last_became_peered":"2024-04-23T01:54:15.631530+0000","last_unstale":"2024-04-23T01:54:36.433649+0000","last_undegraded":"2024-04-23T01:54:36.433649+0000","last_fullsized":"2024-04-23T01:54:36.433649+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":134,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.712297+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:54:35.712297+0000","last_peered":"2024-04-23T01:54:35.712297+0000","last_clean":"2024-04-23T01:54:35.712297+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:54:35.712297+0000","last_undegraded":"2024-04-23T01:54:35.712297+0000","last_fullsized":"2024-04-23T01:54:35.712297+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"60'215","reported_seq":246,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:36.486849+0000","last_change":"2024-04-23T01:54:15.617410+0000","last_active":"2024-04-23T01:54:36.486849+0000","last_peered":"2024-04-23T01:54:36.486849+0000","last_clean":"2024-04-23T01:54:36.486849+0000","last_became_active":"2024-04-23T01:54:15.610208+0000","last_became_peered":"2024-04-23T01:54:15.610208+0000","last_unstale":"2024-04-23T01:54:36.486849+0000","last_undegraded":"2024-04-23T01:54:36.486849+0000","last_fullsized":"2024-04-23T01:54:36.486849+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1769472,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":216,"num_write_kb":1728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":136,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:33.278089+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:54:33.278089+0000","last_peered":"2024-04-23T01:54:33.278089+0000","last_clean":"2024-04-23T01:54:33.278089+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:54:33.278089+0000","last_undegraded":"2024-04-23T01:54:33.278089+0000","last_fullsized":"2024-04-23T01:54:33.278089+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"60'119","reported_seq":150,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:36.601238+0000","last_change":"2024-04-23T01:54:15.630052+0000","last_active":"2024-04-23T01:54:36.601238+0000","last_peered":"2024-04-23T01:54:36.601238+0000","last_clean":"2024-04-23T01:54:36.601238+0000","last_became_active":"2024-04-23T01:54:15.624615+0000","last_became_peered":"2024-04-23T01:54:15.624615+0000","last_unstale":"2024-04-23T01:54:36.601238+0000","last_undegraded":"2024-04-23T01:54:36.601238+0000","last_fullsized":"2024-04-23T01:54:36.601238+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":138,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.658618+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:54:35.658618+0000","last_peered":"2024-04-23T01:54:35.658618+0000","last_clean":"2024-04-23T01:54:35.658618+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:54:35.658618+0000","last_undegraded":"2024-04-23T01:54:35.658618+0000","last_fullsized":"2024-04-23T01:54:35.658618+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":23830528,"num_objects":365,"num_object_clones":0,"num_object_copies":1095,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2909,"num_write_kb":23272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":35753984,"data_stored":71507968,"data_compressed":3689037,"data_compressed_allocated":35753984,"data_compressed_original":71507968,"omap_allocated":0,"internal_metadata":0},"log_size":2895,"ondisk_log_size":2895,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706233,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":103064,"kb_used_data":9616,"kb_used_omap":0,"kb_used_meta":93440,"kb_avail":94268776,"statfs":{"total":96636764160,"available":96531226624,"internally_reserved":0,"allocated":9846784,"data_stored":19314703,"data_compressed":986352,"data_compressed_allocated":9310208,"data_compressed_original":19111936,"omap_allocated":0,"internal_metadata":95682560},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706233,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":100184,"kb_used_data":6736,"kb_used_omap":0,"kb_used_meta":93440,"kb_avail":94271656,"statfs":{"total":96636764160,"available":96534175744,"internally_reserved":0,"allocated":6897664,"data_stored":13416463,"data_compressed":682054,"data_compressed_allocated":6361088,"data_compressed_original":13213696,"omap_allocated":0,"internal_metadata":95682560},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706233,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":95880,"kb_used_data":10752,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94275960,"statfs":{"total":96636764160,"available":96538583040,"internally_reserved":0,"allocated":11010048,"data_stored":21173743,"data_compressed":1082712,"data_compressed_allocated":10493952,"data_compressed_original":20987904,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706233,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":102796,"kb_used_data":9348,"kb_used_omap":0,"kb_used_meta":93440,"kb_avail":94269044,"statfs":{"total":96636764160,"available":96531501056,"internally_reserved":0,"allocated":9572352,"data_stored":18765839,"data_compressed":957983,"data_compressed_allocated":9035776,"data_compressed_original":18563072,"omap_allocated":0,"internal_metadata":95682560},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9101312,"data_stored":18202624,"data_compressed":939028,"data_compressed_allocated":9101312,"data_compressed_original":18202624,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10686464,"data_stored":21372928,"data_compressed":1102593,"data_compressed_allocated":10686464,"data_compressed_original":21372928,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6553600,"data_stored":13107200,"data_compressed":676212,"data_compressed_allocated":6553600,"data_compressed_original":13107200,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9412608,"data_stored":18825216,"data_compressed":971204,"data_compressed_allocated":9412608,"data_compressed_original":18825216,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:54:40.520 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 24 16 3466 3450 1.12295 1.05469 0.249918 0.111019 2024-04-23T01:54:40.531 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:54:40.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:40.652+0000 7f1ab4806640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:40.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:40.659+0000 7f1ab4806640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:41.521 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 25 16 3603 3587 1.12084 1.07031 0.195197 0.111289 2024-04-23T01:54:42.521 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 26 16 3742 3726 1.1195 1.08594 0.148624 0.111353 2024-04-23T01:54:42.871 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:54:42.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:42.991+0000 7f854caad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:43.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:42.999+0000 7f854caad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:43.522 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 27 16 3873 3857 1.11593 1.02344 0.044221 0.11181 2024-04-23T01:54:44.521 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 28 16 4012 3996 1.11486 1.08594 0.14445 0.111821 2024-04-23T01:54:44.967 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:54:44.967 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:54:44.967 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:54:44.967 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:54:45.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:45.086+0000 7f763cf9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:45.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:45.098+0000 7f763cf9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:45.133 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:54:45.141 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:54:45.141 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:54:45.141 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:54:45.141 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:54:45.141 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:54:45.141 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:54:45.141 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:54:45.141 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:54:45.141 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:54:45.141 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:54:45.142 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:54:45.142 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:54:45.145 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:54:45.146 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:54:45.146 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:54:45.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:45.269+0000 7f47ecc6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:45.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:45.280+0000 7f47ecc6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:45.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:45.317+0000 7f2d09820640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:45.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:45.331+0000 7f2d09820640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:45.363 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:54:45.363 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:54:45.363 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:54:45.363 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:54:45.363 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:54:45.363 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:54:45.363 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:54:45.363 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:54:45.363 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:54:45.363 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:54:45.363 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:54:45.363 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:54:45.367 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:54:45.367 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:54:45.368 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:54:45.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:45.487+0000 7f21a026d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:45.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:45.499+0000 7f21a026d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:45.521 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 29 16 4149 4133 1.11332 1.07031 0.218386 0.11189 2024-04-23T01:54:45.522 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:54:45.522 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:54:45.522 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:54:45.522 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:54:45.522 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:54:45.522 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:54:45.522 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:54:45.522 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:54:45.522 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:54:45.522 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:54:45.522 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:54:45.523 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:54:45.525 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:54:45.525 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:54:45.526 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:54:45.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:45.648+0000 7f59dbaed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:45.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:45.661+0000 7f59dbaed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:45.704 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:54:45.704 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:54:45.704 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:54:45.704 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:54:45.704 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:54:45.704 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:54:45.704 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:54:45.704 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:54:45.705 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:54:45.705 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:54:45.705 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:54:45.705 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:54:45.708 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:54:45.708 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:54:46.521 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 30 16 4289 4273 1.11266 1.09375 0.162976 0.112071 2024-04-23T01:54:47.444 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:54:47.521 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 31 16 4436 4420 1.11381 1.14844 0.133488 0.111976 2024-04-23T01:54:47.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:47.574+0000 7f18c8c2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:47.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:47.580+0000 7f18c8c2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:48.521 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 32 16 4565 4549 1.1105 1.00781 0.17707 0.112306 2024-04-23T01:54:49.521 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 33 16 4713 4697 1.11188 1.15625 0.10397 0.112107 2024-04-23T01:54:49.766 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:54:49.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:49.890+0000 7f5a38954640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:49.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:49.898+0000 7f5a38954640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:50.521 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 34 16 4848 4832 1.1102 1.05469 0.0960558 0.112353 2024-04-23T01:54:50.709 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:54:50.709 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:54:50.709 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:54:50.709 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:54:50.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:50.837+0000 7f1e1322d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:50.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:50.851+0000 7f1e1322d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:50.889 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:54:50.890 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:54:50.890 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:54:50.890 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:54:50.890 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:54:50.890 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:54:50.890 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:54:50.890 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:54:50.890 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:54:50.890 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:54:50.890 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:54:50.890 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:54:50.893 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:54:50.893 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:54:50.894 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:54:51.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:51.018+0000 7fb5fbe2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:51.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:51.029+0000 7fb5fbe2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:51.049 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:54:51.049 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:54:51.049 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:54:51.049 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:54:51.049 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:54:51.049 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:54:51.049 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:54:51.049 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:54:51.049 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:54:51.050 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:54:51.050 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:54:51.050 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:54:51.052 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:54:51.053 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:54:51.053 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:54:51.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:51.170+0000 7f8f2555d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:51.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:51.181+0000 7f8f2555d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:51.205 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:54:51.205 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:54:51.206 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:54:51.206 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:54:51.206 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:54:51.206 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:54:51.206 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:54:51.206 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:54:51.206 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:54:51.206 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:54:51.206 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:54:51.206 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:54:51.208 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:54:51.209 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:54:51.209 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:54:51.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:51.330+0000 7f71f126d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:51.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:51.341+0000 7f71f126d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:51.366 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:54:51.366 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:54:51.366 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:54:51.366 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:54:51.366 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:54:51.366 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:54:51.366 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:54:51.366 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:54:51.366 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:54:51.366 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:54:51.367 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:54:51.367 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:54:51.368 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:54:51.369 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:54:51.521 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 35 16 4993 4977 1.11084 1.13281 0.0748749 0.112275 2024-04-23T01:54:52.213 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:54:52.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:52.341+0000 7f9ff6f5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:52.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:52.349+0000 7f9ff6f5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:52.522 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 36 16 5132 5116 1.11015 1.08594 0.0593058 0.112383 2024-04-23T01:54:53.522 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 37 16 5265 5249 1.10822 1.03906 0.118805 0.112558 2024-04-23T01:54:54.495 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:54:54.522 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 38 16 5399 5383 1.1066 1.04688 0.0931503 0.112758 2024-04-23T01:54:54.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:54.615+0000 7fd585aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:54.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:54.621+0000 7fd585aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:55.522 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 39 16 5524 5508 1.10327 0.976562 0.117777 0.113101 2024-04-23T01:54:56.369 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:54:56.369 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:54:56.369 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:54:56.370 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:54:56.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:56.493+0000 7fd9c686d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:56.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:56.505+0000 7fd9c686d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:56.522 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:54:56.522089+0000 min lat: 0.015902 max lat: 0.347473 avg lat: 0.113061 2024-04-23T01:54:56.522 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:54:56.522 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 40 16 5666 5650 1.10342 1.10938 0.0525434 0.113061 2024-04-23T01:54:56.561 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:54:56.561 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:54:56.561 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:54:56.561 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:54:56.561 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:54:56.561 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:54:56.561 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:54:56.562 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:54:56.562 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:54:56.562 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:54:56.562 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:54:56.562 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:54:56.564 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:54:56.565 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:54:56.565 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:54:56.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:56.691+0000 7f6fad15d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:56.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:56.704+0000 7f6fad15d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:56.747 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:54:56.747 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:54:56.747 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:54:56.747 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:54:56.747 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:54:56.747 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:54:56.747 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:54:56.747 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:54:56.747 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:54:56.748 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:54:56.748 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:54:56.748 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:54:56.750 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:54:56.751 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:54:56.751 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:54:56.844 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:54:56.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:56.912+0000 7f8d3599d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:56.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:56.931+0000 7f8d3599d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:56.961 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:54:56.961 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:54:56.961 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:54:56.961 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:54:56.961 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:54:56.961 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:54:56.961 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:54:56.962 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:54:56.962 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:54:56.962 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:54:56.962 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:54:56.962 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:54:56.967 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:54:56.967 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:54:56.967 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:54:56.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:56.983+0000 7f438dd9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:56.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:56.993+0000 7f438dd9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:57.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:57.111+0000 7fbee8f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:57.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:57.123+0000 7fbee8f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:57.149 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:54:57.149 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:54:57.149 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:54:57.149 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:54:57.149 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:54:57.149 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:54:57.150 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:54:57.150 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:54:57.150 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:54:57.150 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:54:57.150 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:54:57.150 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:54:57.152 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:54:57.153 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:54:57.522 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 41 16 5807 5791 1.10337 1.10156 0.122851 0.113096 2024-04-23T01:54:58.522 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 42 16 5950 5934 1.1037 1.11719 0.131691 0.11304 2024-04-23T01:54:59.220 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:54:59.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:59.342+0000 7fcb9466d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:59.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:54:59.349+0000 7fcb9466d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:54:59.522 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 43 16 6081 6065 1.10183 1.02344 0.18992 0.113131 2024-04-23T01:55:00.522 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 44 16 6223 6207 1.102 1.10938 0.0971743 0.113283 2024-04-23T01:55:01.522 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 45 16 6363 6347 1.10181 1.09375 0.157544 0.11326 2024-04-23T01:55:01.542 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:55:01.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:01.682+0000 7fb68366d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:01.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:01.690+0000 7fb68366d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:02.154 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:55:02.154 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:55:02.154 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:55:02.155 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:55:02.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:02.277+0000 7f0198e20640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:02.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:02.290+0000 7f0198e20640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:02.317 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:55:02.317 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:55:02.318 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:55:02.318 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:55:02.318 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:55:02.318 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:55:02.318 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:55:02.318 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:55:02.318 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:55:02.318 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:55:02.318 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:55:02.318 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:55:02.320 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:55:02.321 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:55:02.321 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:55:02.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:02.465+0000 7f763ad9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:02.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:02.477+0000 7f763ad9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:02.513 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:55:02.513 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:55:02.514 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:55:02.514 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:55:02.514 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:55:02.514 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:55:02.514 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:55:02.514 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:55:02.514 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:55:02.514 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:55:02.514 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:55:02.514 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:55:02.516 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:55:02.516 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:55:02.517 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:55:02.522 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 46 16 6482 6466 1.09807 0.929688 0.0552309 0.113648 2024-04-23T01:55:02.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:02.637+0000 7faff519d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:02.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:02.646+0000 7faff519d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:02.665 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:55:02.665 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:55:02.665 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:55:02.665 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:55:02.665 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:55:02.665 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:55:02.665 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:55:02.665 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:55:02.665 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:55:02.665 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:55:02.665 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:55:02.665 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:55:02.668 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:55:02.668 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:55:02.668 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:55:02.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:02.854+0000 7f171d26d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:02.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:02.866+0000 7f171d26d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:02.939 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:55:02.939 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:55:02.939 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:55:02.939 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:55:02.939 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:55:02.939 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:55:02.939 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:55:02.939 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:55:02.939 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:55:02.939 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:55:02.939 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:55:02.939 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:55:02.941 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:55:02.942 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:55:03.523 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 47 16 6629 6613 1.09914 1.14844 0.0930716 0.113589 2024-04-23T01:55:03.874 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:55:04.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:04.007+0000 7f63fbc6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:04.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:04.014+0000 7f63fbc6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:04.523 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 48 16 6764 6748 1.09821 1.05469 0.124693 0.113629 2024-04-23T01:55:05.523 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 49 16 6887 6871 1.09541 0.960938 0.199428 0.113907 2024-04-23T01:55:06.233 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:55:06.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:06.362+0000 7f597666d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:06.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:06.368+0000 7f597666d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:06.523 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 50 16 7021 7005 1.09443 1.04688 0.20155 0.114037 2024-04-23T01:55:07.523 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 51 16 7167 7151 1.09534 1.14062 0.0374891 0.113962 2024-04-23T01:55:07.943 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:55:07.943 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:55:07.943 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:55:07.944 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:55:08.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:08.081+0000 7f68e506d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:08.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:08.092+0000 7f68e506d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:08.119 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:55:08.119 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:55:08.119 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:55:08.119 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:55:08.119 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:55:08.120 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:55:08.120 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:55:08.120 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:55:08.120 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:55:08.120 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:55:08.120 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:55:08.120 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:55:08.122 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:55:08.122 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:55:08.123 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:55:08.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:08.252+0000 7f08dc55d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:08.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:08.264+0000 7f08dc55d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:08.364 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:55:08.364 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:55:08.364 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:55:08.364 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:55:08.364 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:55:08.364 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:55:08.364 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:55:08.364 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:55:08.364 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:55:08.364 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:55:08.365 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:55:08.365 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:55:08.367 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:55:08.368 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:55:08.368 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:55:08.523 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 52 16 7317 7301 1.09681 1.17188 0.0854272 0.113812 2024-04-23T01:55:08.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:08.523+0000 7f3326606640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:08.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:08.535+0000 7f3326606640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:08.554 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:55:08.554 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:55:08.554 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:55:08.554 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:55:08.555 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:55:08.555 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:55:08.555 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:55:08.555 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:55:08.555 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:55:08.555 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:55:08.555 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:55:08.555 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:55:08.557 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:55:08.558 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:55:08.558 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:55:08.608 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:55:08.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:08.683+0000 7fe9f706d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:08.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:08.697+0000 7fe9f706d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:08.724 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:55:08.724 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:55:08.724 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:55:08.724 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:55:08.724 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:55:08.724 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:55:08.724 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:55:08.724 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:55:08.724 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:55:08.724 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:55:08.724 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:55:08.725 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:55:08.731 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:55:08.731 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:55:08.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:08.750+0000 7f26dbead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:08.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:08.758+0000 7f26dbead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:09.523 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 53 16 7440 7424 1.09424 0.960938 0.0777543 0.114062 2024-04-23T01:55:10.523 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 54 16 7569 7553 1.09264 1.00781 0.224194 0.114222 2024-04-23T01:55:10.964 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:55:11.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:11.092+0000 7eff07824640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:11.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:11.099+0000 7eff07824640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:11.523 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 55 16 7707 7691 1.09237 1.07812 0.0705496 0.114287 2024-04-23T01:55:12.524 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 56 16 7854 7838 1.09337 1.14844 0.112329 0.114158 2024-04-23T01:55:13.307 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:55:13.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:13.432+0000 7f7e3931d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:13.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:13.439+0000 7f7e3931d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:13.524 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 57 16 7994 7978 1.09338 1.09375 0.107402 0.114172 2024-04-23T01:55:13.731 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:55:13.732 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:55:13.732 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:55:13.732 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:55:13.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:13.850+0000 7f5c12aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:13.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:13.862+0000 7f5c12aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:13.987 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:55:13.987 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:55:13.987 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:55:13.987 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:55:13.987 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:55:13.987 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:55:13.987 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:55:13.987 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:55:13.987 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:55:13.987 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:55:13.987 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:55:13.987 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:55:13.990 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:55:13.991 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:55:13.991 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:55:14.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:14.136+0000 7fb62775d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:14.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:14.151+0000 7fb62775d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:14.222 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:55:14.222 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:55:14.223 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:55:14.223 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:55:14.223 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:55:14.223 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:55:14.223 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:55:14.223 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:55:14.223 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:55:14.223 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:55:14.223 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:55:14.223 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:55:14.225 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:55:14.226 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:55:14.226 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:55:14.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:14.349+0000 7fd2230ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:14.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:14.361+0000 7fd2230ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:14.380 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:55:14.380 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:55:14.380 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:55:14.380 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:55:14.380 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:55:14.380 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:55:14.381 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:55:14.381 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:55:14.381 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:55:14.381 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:55:14.381 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:55:14.381 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:55:14.384 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:55:14.385 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:55:14.385 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:55:14.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:14.508+0000 7f63d042d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:14.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:14.520+0000 7f63d042d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:14.523 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 58 16 8137 8121 1.09379 1.11719 0.052024 0.114123 2024-04-23T01:55:14.553 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:55:14.553 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:55:14.553 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:55:14.554 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:55:14.554 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:55:14.554 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:55:14.554 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:55:14.554 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:55:14.554 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:55:14.554 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:55:14.554 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:55:14.554 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:55:14.557 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:55:14.558 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:55:15.524 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 59 16 8282 8266 1.09445 1.13281 0.0648424 0.114056 2024-04-23T01:55:15.644 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:55:15.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:15.770+0000 7f2b54e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:15.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:15.776+0000 7f2b54e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:16.524 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:55:16.523939+0000 min lat: 0.0153179 max lat: 0.466783 avg lat: 0.114228 2024-04-23T01:55:16.524 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:55:16.524 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 60 16 8409 8393 1.09274 0.992188 0.119678 0.114228 2024-04-23T01:55:17.524 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 61 16 8539 8523 1.09148 1.01562 0.128371 0.114381 2024-04-23T01:55:18.037 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:55:18.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:18.151+0000 7f32dce2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:18.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:18.158+0000 7f32dce2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:18.524 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 62 16 8659 8643 1.08899 0.9375 0.181014 0.114607 2024-04-23T01:55:19.524 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 63 16 8791 8775 1.08807 1.03125 0.126822 0.114697 2024-04-23T01:55:19.558 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:55:19.559 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:55:19.559 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:55:19.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:19.692+0000 7f5e250ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:19.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:19.702+0000 7f5e250ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:20.017 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:55:20.018 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:55:20.039 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-04-23T01:55:19.666180+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":69550624,"num_objects":1056,"num_object_clones":0,"num_object_copies":3168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":8555,"num_write_kb":69238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8499,"ondisk_log_size":8499,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":505788,"kb_used_data":102812,"kb_used_omap":0,"kb_used_meta":402944,"kb_avail":376981572,"statfs":{"total":386547056640,"available":386029129728,"internally_reserved":0,"allocated":105279488,"data_stored":208576028,"data_compressed":10726671,"data_compressed_allocated":103153664,"data_compressed_original":207781888,"omap_allocated":0,"internal_metadata":412614656},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11370496,"num_objects":173,"num_object_clones":0,"num_object_copies":519,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1388,"num_write_kb":11104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002264"},"pg_stats":[{"pgid":"8.7","version":"60'567","reported_seq":598,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:14.570549+0000","last_change":"2024-04-23T01:54:15.581616+0000","last_active":"2024-04-23T01:55:14.570549+0000","last_peered":"2024-04-23T01:55:14.570549+0000","last_clean":"2024-04-23T01:55:14.570549+0000","last_became_active":"2024-04-23T01:54:15.576738+0000","last_became_peered":"2024-04-23T01:54:15.576738+0000","last_unstale":"2024-04-23T01:55:14.570549+0000","last_undegraded":"2024-04-23T01:55:14.570549+0000","last_fullsized":"2024-04-23T01:55:14.570549+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":568,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"60'523","reported_seq":554,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:16.705127+0000","last_change":"2024-04-23T01:54:15.596716+0000","last_active":"2024-04-23T01:55:16.705127+0000","last_peered":"2024-04-23T01:55:16.705127+0000","last_clean":"2024-04-23T01:55:16.705127+0000","last_became_active":"2024-04-23T01:54:15.589907+0000","last_became_peered":"2024-04-23T01:54:15.589907+0000","last_unstale":"2024-04-23T01:55:16.705127+0000","last_undegraded":"2024-04-23T01:55:16.705127+0000","last_fullsized":"2024-04-23T01:55:16.705127+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4292608,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":524,"num_write_kb":4192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"60'439","reported_seq":472,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:16.171937+0000","last_change":"2024-04-23T01:54:15.700282+0000","last_active":"2024-04-23T01:55:16.171937+0000","last_peered":"2024-04-23T01:55:16.171937+0000","last_clean":"2024-04-23T01:55:16.171937+0000","last_became_active":"2024-04-23T01:54:15.686441+0000","last_became_peered":"2024-04-23T01:54:15.686441+0000","last_unstale":"2024-04-23T01:55:16.171937+0000","last_undegraded":"2024-04-23T01:55:16.171937+0000","last_fullsized":"2024-04-23T01:55:16.171937+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3604480,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":440,"num_write_kb":3520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"60'540","reported_seq":571,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:16.611631+0000","last_change":"2024-04-23T01:54:15.681870+0000","last_active":"2024-04-23T01:55:16.611631+0000","last_peered":"2024-04-23T01:55:16.611631+0000","last_clean":"2024-04-23T01:55:16.611631+0000","last_became_active":"2024-04-23T01:54:15.677678+0000","last_became_peered":"2024-04-23T01:54:15.677678+0000","last_unstale":"2024-04-23T01:55:16.611631+0000","last_undegraded":"2024-04-23T01:55:16.611631+0000","last_fullsized":"2024-04-23T01:55:16.611631+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4431872,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":541,"num_write_kb":4328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3","version":"60'519","reported_seq":552,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:14.284209+0000","last_change":"2024-04-23T01:54:15.711951+0000","last_active":"2024-04-23T01:55:14.284209+0000","last_peered":"2024-04-23T01:55:14.284209+0000","last_clean":"2024-04-23T01:55:14.284209+0000","last_became_active":"2024-04-23T01:54:15.696611+0000","last_became_peered":"2024-04-23T01:54:15.696611+0000","last_unstale":"2024-04-23T01:55:14.284209+0000","last_undegraded":"2024-04-23T01:55:14.284209+0000","last_fullsized":"2024-04-23T01:55:14.284209+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259840,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":520,"num_write_kb":4160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"60'535","reported_seq":566,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:16.147531+0000","last_change":"2024-04-23T01:54:15.676654+0000","last_active":"2024-04-23T01:55:16.147531+0000","last_peered":"2024-04-23T01:55:16.147531+0000","last_clean":"2024-04-23T01:55:16.147531+0000","last_became_active":"2024-04-23T01:54:15.672239+0000","last_became_peered":"2024-04-23T01:54:15.672239+0000","last_unstale":"2024-04-23T01:55:16.147531+0000","last_undegraded":"2024-04-23T01:55:16.147531+0000","last_fullsized":"2024-04-23T01:55:16.147531+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4390912,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":536,"num_write_kb":4288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"60'519","reported_seq":552,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:15.068720+0000","last_change":"2024-04-23T01:54:15.609601+0000","last_active":"2024-04-23T01:55:15.068720+0000","last_peered":"2024-04-23T01:55:15.068720+0000","last_clean":"2024-04-23T01:55:15.068720+0000","last_became_active":"2024-04-23T01:54:15.598424+0000","last_became_peered":"2024-04-23T01:54:15.598424+0000","last_unstale":"2024-04-23T01:55:15.068720+0000","last_undegraded":"2024-04-23T01:55:15.068720+0000","last_fullsized":"2024-04-23T01:55:15.068720+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259840,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":520,"num_write_kb":4160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"60'623","reported_seq":656,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:16.473425+0000","last_change":"2024-04-23T01:54:15.718344+0000","last_active":"2024-04-23T01:55:16.473425+0000","last_peered":"2024-04-23T01:55:16.473425+0000","last_clean":"2024-04-23T01:55:16.473425+0000","last_became_active":"2024-04-23T01:54:15.706071+0000","last_became_peered":"2024-04-23T01:54:15.706071+0000","last_unstale":"2024-04-23T01:55:16.473425+0000","last_undegraded":"2024-04-23T01:55:16.473425+0000","last_fullsized":"2024-04-23T01:55:16.473425+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5111808,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":624,"num_write_kb":4992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"60'519","reported_seq":550,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:16.226891+0000","last_change":"2024-04-23T01:54:15.596696+0000","last_active":"2024-04-23T01:55:16.226891+0000","last_peered":"2024-04-23T01:55:16.226891+0000","last_clean":"2024-04-23T01:55:16.226891+0000","last_became_active":"2024-04-23T01:54:15.588060+0000","last_became_peered":"2024-04-23T01:54:15.588060+0000","last_unstale":"2024-04-23T01:55:16.226891+0000","last_undegraded":"2024-04-23T01:55:16.226891+0000","last_fullsized":"2024-04-23T01:55:16.226891+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259840,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":520,"num_write_kb":4160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":267,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.696720+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:54:35.696720+0000","last_peered":"2024-04-23T01:54:35.696720+0000","last_clean":"2024-04-23T01:54:35.696720+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:54:35.696720+0000","last_undegraded":"2024-04-23T01:54:35.696720+0000","last_fullsized":"2024-04-23T01:54:35.696720+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":138,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.679304+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:54:35.679304+0000","last_peered":"2024-04-23T01:54:35.679304+0000","last_clean":"2024-04-23T01:54:35.679304+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:54:35.679304+0000","last_undegraded":"2024-04-23T01:54:35.679304+0000","last_fullsized":"2024-04-23T01:54:35.679304+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"60'463","reported_seq":496,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:15.912749+0000","last_change":"2024-04-23T01:54:15.705667+0000","last_active":"2024-04-23T01:55:15.912749+0000","last_peered":"2024-04-23T01:55:15.912749+0000","last_clean":"2024-04-23T01:55:15.912749+0000","last_became_active":"2024-04-23T01:54:15.701510+0000","last_became_peered":"2024-04-23T01:54:15.701510+0000","last_unstale":"2024-04-23T01:55:15.912749+0000","last_undegraded":"2024-04-23T01:55:15.912749+0000","last_fullsized":"2024-04-23T01:55:15.912749+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":464,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":130,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.677386+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:54:35.677386+0000","last_peered":"2024-04-23T01:54:35.677386+0000","last_clean":"2024-04-23T01:54:35.677386+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:54:35.677386+0000","last_undegraded":"2024-04-23T01:54:35.677386+0000","last_fullsized":"2024-04-23T01:54:35.677386+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"60'540","reported_seq":571,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:16.598894+0000","last_change":"2024-04-23T01:54:15.668120+0000","last_active":"2024-04-23T01:55:16.598894+0000","last_peered":"2024-04-23T01:55:16.598894+0000","last_clean":"2024-04-23T01:55:16.598894+0000","last_became_active":"2024-04-23T01:54:15.661561+0000","last_became_peered":"2024-04-23T01:54:15.661561+0000","last_unstale":"2024-04-23T01:55:16.598894+0000","last_undegraded":"2024-04-23T01:55:16.598894+0000","last_fullsized":"2024-04-23T01:55:16.598894+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4431872,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":541,"num_write_kb":4328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":138,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.657300+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:54:35.657300+0000","last_peered":"2024-04-23T01:54:35.657300+0000","last_clean":"2024-04-23T01:54:35.657300+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:54:35.657300+0000","last_undegraded":"2024-04-23T01:54:35.657300+0000","last_fullsized":"2024-04-23T01:54:35.657300+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"60'459","reported_seq":492,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:16.645066+0000","last_change":"2024-04-23T01:54:15.677434+0000","last_active":"2024-04-23T01:55:16.645066+0000","last_peered":"2024-04-23T01:55:16.645066+0000","last_clean":"2024-04-23T01:55:16.645066+0000","last_became_active":"2024-04-23T01:54:15.670406+0000","last_became_peered":"2024-04-23T01:54:15.670406+0000","last_unstale":"2024-04-23T01:55:16.645066+0000","last_undegraded":"2024-04-23T01:55:16.645066+0000","last_fullsized":"2024-04-23T01:55:16.645066+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3768320,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":460,"num_write_kb":3680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":130,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.679268+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:54:35.679268+0000","last_peered":"2024-04-23T01:54:35.679268+0000","last_clean":"2024-04-23T01:54:35.679268+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:54:35.679268+0000","last_undegraded":"2024-04-23T01:54:35.679268+0000","last_fullsized":"2024-04-23T01:54:35.679268+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.e","version":"60'487","reported_seq":518,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:15.267758+0000","last_change":"2024-04-23T01:54:15.652153+0000","last_active":"2024-04-23T01:55:15.267758+0000","last_peered":"2024-04-23T01:55:15.267758+0000","last_clean":"2024-04-23T01:55:15.267758+0000","last_became_active":"2024-04-23T01:54:15.644332+0000","last_became_peered":"2024-04-23T01:54:15.644332+0000","last_unstale":"2024-04-23T01:55:15.267758+0000","last_undegraded":"2024-04-23T01:55:15.267758+0000","last_fullsized":"2024-04-23T01:55:15.267758+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3997696,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":488,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":136,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:33.277036+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:54:33.277036+0000","last_peered":"2024-04-23T01:54:33.277036+0000","last_clean":"2024-04-23T01:54:33.277036+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:54:33.277036+0000","last_undegraded":"2024-04-23T01:54:33.277036+0000","last_fullsized":"2024-04-23T01:54:33.277036+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"60'631","reported_seq":664,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:16.385813+0000","last_change":"2024-04-23T01:54:15.637842+0000","last_active":"2024-04-23T01:55:16.385813+0000","last_peered":"2024-04-23T01:55:16.385813+0000","last_clean":"2024-04-23T01:55:16.385813+0000","last_became_active":"2024-04-23T01:54:15.631530+0000","last_became_peered":"2024-04-23T01:54:15.631530+0000","last_unstale":"2024-04-23T01:55:16.385813+0000","last_undegraded":"2024-04-23T01:55:16.385813+0000","last_fullsized":"2024-04-23T01:55:16.385813+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":5056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":134,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.712297+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:54:35.712297+0000","last_peered":"2024-04-23T01:54:35.712297+0000","last_clean":"2024-04-23T01:54:35.712297+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:54:35.712297+0000","last_undegraded":"2024-04-23T01:54:35.712297+0000","last_fullsized":"2024-04-23T01:54:35.712297+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"60'543","reported_seq":574,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:16.060343+0000","last_change":"2024-04-23T01:54:15.617410+0000","last_active":"2024-04-23T01:55:16.060343+0000","last_peered":"2024-04-23T01:55:16.060343+0000","last_clean":"2024-04-23T01:55:16.060343+0000","last_became_active":"2024-04-23T01:54:15.610208+0000","last_became_peered":"2024-04-23T01:54:15.610208+0000","last_unstale":"2024-04-23T01:55:16.060343+0000","last_undegraded":"2024-04-23T01:55:16.060343+0000","last_fullsized":"2024-04-23T01:55:16.060343+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":544,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":136,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:33.278089+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:54:33.278089+0000","last_peered":"2024-04-23T01:54:33.278089+0000","last_clean":"2024-04-23T01:54:33.278089+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:54:33.278089+0000","last_undegraded":"2024-04-23T01:54:33.278089+0000","last_fullsized":"2024-04-23T01:54:33.278089+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"60'495","reported_seq":526,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:16.581780+0000","last_change":"2024-04-23T01:54:15.630052+0000","last_active":"2024-04-23T01:55:16.581780+0000","last_peered":"2024-04-23T01:55:16.581780+0000","last_clean":"2024-04-23T01:55:16.581780+0000","last_became_active":"2024-04-23T01:54:15.624615+0000","last_became_peered":"2024-04-23T01:54:15.624615+0000","last_unstale":"2024-04-23T01:55:16.581780+0000","last_undegraded":"2024-04-23T01:55:16.581780+0000","last_fullsized":"2024-04-23T01:55:16.581780+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4063232,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":496,"num_write_kb":3968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":138,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.658618+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:54:35.658618+0000","last_peered":"2024-04-23T01:54:35.658618+0000","last_clean":"2024-04-23T01:54:35.658618+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:54:35.658618+0000","last_undegraded":"2024-04-23T01:54:35.658618+0000","last_fullsized":"2024-04-23T01:54:35.658618+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":68960256,"num_objects":1054,"num_object_clones":0,"num_object_copies":3162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8418,"num_write_kb":67344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":103452672,"data_stored":206905344,"data_compressed":10680381,"data_compressed_allocated":103452672,"data_compressed_original":206905344,"omap_allocated":0,"internal_metadata":0},"log_size":8402,"ondisk_log_size":8402,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706273,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":137664,"kb_used_data":26168,"kb_used_omap":0,"kb_used_meta":111488,"kb_avail":94234176,"statfs":{"total":96636764160,"available":96495796224,"internally_reserved":0,"allocated":26796032,"data_stored":53213199,"data_compressed":2736726,"data_compressed_allocated":26259456,"data_compressed_original":53010432,"omap_allocated":0,"internal_metadata":114163712},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706273,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":108608,"kb_used_data":19320,"kb_used_omap":0,"kb_used_meta":89280,"kb_avail":94263232,"statfs":{"total":96636764160,"available":96525549568,"internally_reserved":0,"allocated":19783680,"data_stored":39188495,"data_compressed":2012812,"data_compressed_allocated":19247104,"data_compressed_original":38985728,"omap_allocated":0,"internal_metadata":91422720},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706273,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":133012,"kb_used_data":29836,"kb_used_omap":0,"kb_used_meta":103168,"kb_avail":94238828,"statfs":{"total":96636764160,"available":96500559872,"internally_reserved":0,"allocated":30552064,"data_stored":60257775,"data_compressed":3100845,"data_compressed_allocated":30035968,"data_compressed_original":60071936,"omap_allocated":0,"internal_metadata":105644032},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706273,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":126504,"kb_used_data":27488,"kb_used_omap":0,"kb_used_meta":99008,"kb_avail":94245336,"statfs":{"total":96636764160,"available":96507224064,"internally_reserved":0,"allocated":28147712,"data_stored":55916559,"data_compressed":2876288,"data_compressed_allocated":27611136,"data_compressed_original":55713792,"omap_allocated":0,"internal_metadata":101384192},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":27648000,"data_stored":55296000,"data_compressed":2854372,"data_compressed_allocated":27648000,"data_compressed_original":55296000,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":30150656,"data_stored":60301312,"data_compressed":3112689,"data_compressed_allocated":30150656,"data_compressed_original":60301312,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19357696,"data_stored":38715392,"data_compressed":1998510,"data_compressed_allocated":19357696,"data_compressed_original":38715392,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":26296320,"data_stored":52592640,"data_compressed":2714810,"data_compressed_allocated":26296320,"data_compressed_original":52592640,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:55:20.375 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:55:20.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:20.512+0000 7f505aa2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:20.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:20.521+0000 7f505aa2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:20.524 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 64 16 8933 8917 1.0884 1.10938 0.110921 0.114674 2024-04-23T01:55:21.524 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 65 16 9048 9032 1.08548 0.898438 0.160053 0.115015 2024-04-23T01:55:22.524 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 66 16 9189 9173 1.08572 1.10156 0.1201 0.115017 2024-04-23T01:55:22.812 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:55:22.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:22.939+0000 7fee6b26d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:22.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:22.947+0000 7fee6b26d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:23.524 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 67 16 9319 9303 1.08467 1.01562 0.0840974 0.115122 2024-04-23T01:55:24.525 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 68 16 9441 9425 1.08274 0.953125 0.109059 0.115328 2024-04-23T01:55:25.041 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:55:25.041 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:55:25.041 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:55:25.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:25.176+0000 7f8e1eb9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:25.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:25.185+0000 7f8e1eb9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:25.238 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:55:25.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:25.384+0000 7fe68c35d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:25.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:25.392+0000 7fe68c35d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:25.423 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:55:25.423 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:55:25.440 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-04-23T01:55:23.667022+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":74793504,"num_objects":1136,"num_object_clones":0,"num_object_copies":3408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":9195,"num_write_kb":74358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9139,"ondisk_log_size":9139,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":509260,"kb_used_data":110444,"kb_used_omap":0,"kb_used_meta":398784,"kb_avail":376978100,"statfs":{"total":386547056640,"available":386025574400,"internally_reserved":0,"allocated":113094656,"data_stored":224206364,"data_compressed":11533755,"data_compressed_allocated":110968832,"data_compressed_original":223412224,"omap_allocated":0,"internal_metadata":408354816},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14426112,"num_objects":220,"num_object_clones":0,"num_object_copies":660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1761,"num_write_kb":14088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002351"},"pg_stats":[{"pgid":"8.7","version":"60'599","reported_seq":630,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:20.199928+0000","last_change":"2024-04-23T01:54:15.581616+0000","last_active":"2024-04-23T01:55:20.199928+0000","last_peered":"2024-04-23T01:55:20.199928+0000","last_clean":"2024-04-23T01:55:20.199928+0000","last_became_active":"2024-04-23T01:54:15.576738+0000","last_became_peered":"2024-04-23T01:54:15.576738+0000","last_unstale":"2024-04-23T01:55:20.199928+0000","last_undegraded":"2024-04-23T01:55:20.199928+0000","last_fullsized":"2024-04-23T01:55:20.199928+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"60'562","reported_seq":593,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:21.729722+0000","last_change":"2024-04-23T01:54:15.596716+0000","last_active":"2024-04-23T01:55:21.729722+0000","last_peered":"2024-04-23T01:55:21.729722+0000","last_clean":"2024-04-23T01:55:21.729722+0000","last_became_active":"2024-04-23T01:54:15.589907+0000","last_became_peered":"2024-04-23T01:54:15.589907+0000","last_unstale":"2024-04-23T01:55:21.729722+0000","last_undegraded":"2024-04-23T01:55:21.729722+0000","last_fullsized":"2024-04-23T01:55:21.729722+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4612096,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":563,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"60'487","reported_seq":520,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:20.364052+0000","last_change":"2024-04-23T01:54:15.700282+0000","last_active":"2024-04-23T01:55:20.364052+0000","last_peered":"2024-04-23T01:55:20.364052+0000","last_clean":"2024-04-23T01:55:20.364052+0000","last_became_active":"2024-04-23T01:54:15.686441+0000","last_became_peered":"2024-04-23T01:54:15.686441+0000","last_unstale":"2024-04-23T01:55:20.364052+0000","last_undegraded":"2024-04-23T01:55:20.364052+0000","last_fullsized":"2024-04-23T01:55:20.364052+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3997696,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":488,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"60'620","reported_seq":651,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:21.598350+0000","last_change":"2024-04-23T01:54:15.681870+0000","last_active":"2024-04-23T01:55:21.598350+0000","last_peered":"2024-04-23T01:55:21.598350+0000","last_clean":"2024-04-23T01:55:21.598350+0000","last_became_active":"2024-04-23T01:54:15.677678+0000","last_became_peered":"2024-04-23T01:54:15.677678+0000","last_unstale":"2024-04-23T01:55:21.598350+0000","last_undegraded":"2024-04-23T01:55:21.598350+0000","last_fullsized":"2024-04-23T01:55:21.598350+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5087232,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":621,"num_write_kb":4968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3","version":"60'575","reported_seq":608,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:19.497488+0000","last_change":"2024-04-23T01:54:15.711951+0000","last_active":"2024-04-23T01:55:19.497488+0000","last_peered":"2024-04-23T01:55:19.497488+0000","last_clean":"2024-04-23T01:55:19.497488+0000","last_became_active":"2024-04-23T01:54:15.696611+0000","last_became_peered":"2024-04-23T01:54:15.696611+0000","last_unstale":"2024-04-23T01:55:19.497488+0000","last_undegraded":"2024-04-23T01:55:19.497488+0000","last_fullsized":"2024-04-23T01:55:19.497488+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718592,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":576,"num_write_kb":4608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"60'606","reported_seq":637,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:21.623426+0000","last_change":"2024-04-23T01:54:15.676654+0000","last_active":"2024-04-23T01:55:21.623426+0000","last_peered":"2024-04-23T01:55:21.623426+0000","last_clean":"2024-04-23T01:55:21.623426+0000","last_became_active":"2024-04-23T01:54:15.672239+0000","last_became_peered":"2024-04-23T01:54:15.672239+0000","last_unstale":"2024-04-23T01:55:21.623426+0000","last_undegraded":"2024-04-23T01:55:21.623426+0000","last_fullsized":"2024-04-23T01:55:21.623426+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4972544,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":607,"num_write_kb":4856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"60'559","reported_seq":592,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:21.602419+0000","last_change":"2024-04-23T01:54:15.609601+0000","last_active":"2024-04-23T01:55:21.602419+0000","last_peered":"2024-04-23T01:55:21.602419+0000","last_clean":"2024-04-23T01:55:21.602419+0000","last_became_active":"2024-04-23T01:54:15.598424+0000","last_became_peered":"2024-04-23T01:54:15.598424+0000","last_unstale":"2024-04-23T01:55:21.602419+0000","last_undegraded":"2024-04-23T01:55:21.602419+0000","last_fullsized":"2024-04-23T01:55:21.602419+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4587520,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":560,"num_write_kb":4480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"60'671","reported_seq":704,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:21.212172+0000","last_change":"2024-04-23T01:54:15.718344+0000","last_active":"2024-04-23T01:55:21.212172+0000","last_peered":"2024-04-23T01:55:21.212172+0000","last_clean":"2024-04-23T01:55:21.212172+0000","last_became_active":"2024-04-23T01:54:15.706071+0000","last_became_peered":"2024-04-23T01:54:15.706071+0000","last_unstale":"2024-04-23T01:55:21.212172+0000","last_undegraded":"2024-04-23T01:55:21.212172+0000","last_fullsized":"2024-04-23T01:55:21.212172+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505024,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":672,"num_write_kb":5376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"60'519","reported_seq":550,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:16.226891+0000","last_change":"2024-04-23T01:54:15.596696+0000","last_active":"2024-04-23T01:55:16.226891+0000","last_peered":"2024-04-23T01:55:16.226891+0000","last_clean":"2024-04-23T01:55:16.226891+0000","last_became_active":"2024-04-23T01:54:15.588060+0000","last_became_peered":"2024-04-23T01:54:15.588060+0000","last_unstale":"2024-04-23T01:55:16.226891+0000","last_undegraded":"2024-04-23T01:55:16.226891+0000","last_fullsized":"2024-04-23T01:55:16.226891+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259840,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":520,"num_write_kb":4160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":267,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.696720+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:54:35.696720+0000","last_peered":"2024-04-23T01:54:35.696720+0000","last_clean":"2024-04-23T01:54:35.696720+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:54:35.696720+0000","last_undegraded":"2024-04-23T01:54:35.696720+0000","last_fullsized":"2024-04-23T01:54:35.696720+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":138,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.679304+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:54:35.679304+0000","last_peered":"2024-04-23T01:54:35.679304+0000","last_clean":"2024-04-23T01:54:35.679304+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:54:35.679304+0000","last_undegraded":"2024-04-23T01:54:35.679304+0000","last_fullsized":"2024-04-23T01:54:35.679304+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"60'495","reported_seq":528,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:21.547707+0000","last_change":"2024-04-23T01:54:15.705667+0000","last_active":"2024-04-23T01:55:21.547707+0000","last_peered":"2024-04-23T01:55:21.547707+0000","last_clean":"2024-04-23T01:55:21.547707+0000","last_became_active":"2024-04-23T01:54:15.701510+0000","last_became_peered":"2024-04-23T01:54:15.701510+0000","last_unstale":"2024-04-23T01:55:21.547707+0000","last_undegraded":"2024-04-23T01:55:21.547707+0000","last_fullsized":"2024-04-23T01:55:21.547707+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4063232,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":496,"num_write_kb":3968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":130,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.677386+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:54:35.677386+0000","last_peered":"2024-04-23T01:54:35.677386+0000","last_clean":"2024-04-23T01:54:35.677386+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:54:35.677386+0000","last_undegraded":"2024-04-23T01:54:35.677386+0000","last_fullsized":"2024-04-23T01:54:35.677386+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"60'575","reported_seq":606,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:19.120119+0000","last_change":"2024-04-23T01:54:15.668120+0000","last_active":"2024-04-23T01:55:19.120119+0000","last_peered":"2024-04-23T01:55:19.120119+0000","last_clean":"2024-04-23T01:55:19.120119+0000","last_became_active":"2024-04-23T01:54:15.661561+0000","last_became_peered":"2024-04-23T01:54:15.661561+0000","last_unstale":"2024-04-23T01:55:19.120119+0000","last_undegraded":"2024-04-23T01:55:19.120119+0000","last_fullsized":"2024-04-23T01:55:19.120119+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718592,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":576,"num_write_kb":4608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":138,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.657300+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:54:35.657300+0000","last_peered":"2024-04-23T01:54:35.657300+0000","last_clean":"2024-04-23T01:54:35.657300+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:54:35.657300+0000","last_undegraded":"2024-04-23T01:54:35.657300+0000","last_fullsized":"2024-04-23T01:54:35.657300+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"60'479","reported_seq":512,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:21.340150+0000","last_change":"2024-04-23T01:54:15.677434+0000","last_active":"2024-04-23T01:55:21.340150+0000","last_peered":"2024-04-23T01:55:21.340150+0000","last_clean":"2024-04-23T01:55:21.340150+0000","last_became_active":"2024-04-23T01:54:15.670406+0000","last_became_peered":"2024-04-23T01:54:15.670406+0000","last_unstale":"2024-04-23T01:55:21.340150+0000","last_undegraded":"2024-04-23T01:55:21.340150+0000","last_fullsized":"2024-04-23T01:55:21.340150+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":480,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":130,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.679268+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:54:35.679268+0000","last_peered":"2024-04-23T01:54:35.679268+0000","last_clean":"2024-04-23T01:54:35.679268+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:54:35.679268+0000","last_undegraded":"2024-04-23T01:54:35.679268+0000","last_fullsized":"2024-04-23T01:54:35.679268+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.e","version":"60'519","reported_seq":550,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:20.917094+0000","last_change":"2024-04-23T01:54:15.652153+0000","last_active":"2024-04-23T01:55:20.917094+0000","last_peered":"2024-04-23T01:55:20.917094+0000","last_clean":"2024-04-23T01:55:20.917094+0000","last_became_active":"2024-04-23T01:54:15.644332+0000","last_became_peered":"2024-04-23T01:54:15.644332+0000","last_unstale":"2024-04-23T01:55:20.917094+0000","last_undegraded":"2024-04-23T01:55:20.917094+0000","last_fullsized":"2024-04-23T01:55:20.917094+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259840,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":520,"num_write_kb":4160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":136,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:33.277036+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:54:33.277036+0000","last_peered":"2024-04-23T01:54:33.277036+0000","last_clean":"2024-04-23T01:54:33.277036+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:54:33.277036+0000","last_undegraded":"2024-04-23T01:54:33.277036+0000","last_fullsized":"2024-04-23T01:54:33.277036+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"60'655","reported_seq":688,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:19.529837+0000","last_change":"2024-04-23T01:54:15.637842+0000","last_active":"2024-04-23T01:55:19.529837+0000","last_peered":"2024-04-23T01:55:19.529837+0000","last_clean":"2024-04-23T01:55:19.529837+0000","last_became_active":"2024-04-23T01:54:15.631530+0000","last_became_peered":"2024-04-23T01:54:15.631530+0000","last_unstale":"2024-04-23T01:55:19.529837+0000","last_undegraded":"2024-04-23T01:55:19.529837+0000","last_fullsized":"2024-04-23T01:55:19.529837+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":134,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.712297+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:54:35.712297+0000","last_peered":"2024-04-23T01:54:35.712297+0000","last_clean":"2024-04-23T01:54:35.712297+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:54:35.712297+0000","last_undegraded":"2024-04-23T01:54:35.712297+0000","last_fullsized":"2024-04-23T01:54:35.712297+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"60'575","reported_seq":606,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:20.717936+0000","last_change":"2024-04-23T01:54:15.617410+0000","last_active":"2024-04-23T01:55:20.717936+0000","last_peered":"2024-04-23T01:55:20.717936+0000","last_clean":"2024-04-23T01:55:20.717936+0000","last_became_active":"2024-04-23T01:54:15.610208+0000","last_became_peered":"2024-04-23T01:54:15.610208+0000","last_unstale":"2024-04-23T01:55:20.717936+0000","last_undegraded":"2024-04-23T01:55:20.717936+0000","last_fullsized":"2024-04-23T01:55:20.717936+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718592,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":576,"num_write_kb":4608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":136,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:33.278089+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:54:33.278089+0000","last_peered":"2024-04-23T01:54:33.278089+0000","last_clean":"2024-04-23T01:54:33.278089+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:54:33.278089+0000","last_undegraded":"2024-04-23T01:54:33.278089+0000","last_fullsized":"2024-04-23T01:54:33.278089+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"60'546","reported_seq":577,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:21.633409+0000","last_change":"2024-04-23T01:54:15.630052+0000","last_active":"2024-04-23T01:55:21.633409+0000","last_peered":"2024-04-23T01:55:21.633409+0000","last_clean":"2024-04-23T01:55:21.633409+0000","last_became_active":"2024-04-23T01:54:15.624615+0000","last_became_peered":"2024-04-23T01:54:15.624615+0000","last_unstale":"2024-04-23T01:55:21.633409+0000","last_undegraded":"2024-04-23T01:55:21.633409+0000","last_fullsized":"2024-04-23T01:55:21.633409+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4481024,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":547,"num_write_kb":4376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":138,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.658618+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:54:35.658618+0000","last_peered":"2024-04-23T01:54:35.658618+0000","last_clean":"2024-04-23T01:54:35.658618+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:54:35.658618+0000","last_undegraded":"2024-04-23T01:54:35.658618+0000","last_fullsized":"2024-04-23T01:54:35.658618+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":74203136,"num_objects":1134,"num_object_clones":0,"num_object_copies":3402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9058,"num_write_kb":72464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":111316992,"data_stored":222633984,"data_compressed":11492541,"data_compressed_allocated":111316992,"data_compressed_original":222633984,"omap_allocated":0,"internal_metadata":0},"log_size":9042,"ondisk_log_size":9042,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706278,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":126968,"kb_used_data":27952,"kb_used_omap":0,"kb_used_meta":99008,"kb_avail":94244872,"statfs":{"total":96636764160,"available":96506748928,"internally_reserved":0,"allocated":28622848,"data_stored":56866831,"data_compressed":2925384,"data_compressed_allocated":28086272,"data_compressed_original":56664064,"omap_allocated":0,"internal_metadata":101384192},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706278,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":114104,"kb_used_data":20656,"kb_used_omap":0,"kb_used_meta":93440,"kb_avail":94257736,"statfs":{"total":96636764160,"available":96519921664,"internally_reserved":0,"allocated":21151744,"data_stored":41924623,"data_compressed":2154094,"data_compressed_allocated":20615168,"data_compressed_original":41721856,"omap_allocated":0,"internal_metadata":95682560},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706278,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":139472,"kb_used_data":32136,"kb_used_omap":0,"kb_used_meta":107328,"kb_avail":94232368,"statfs":{"total":96636764160,"available":96493944832,"internally_reserved":0,"allocated":32907264,"data_stored":64968175,"data_compressed":3344070,"data_compressed_allocated":32391168,"data_compressed_original":64782336,"omap_allocated":0,"internal_metadata":109903872},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706278,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":128716,"kb_used_data":29700,"kb_used_omap":0,"kb_used_meta":99008,"kb_avail":94243124,"statfs":{"total":96636764160,"available":96504958976,"internally_reserved":0,"allocated":30412800,"data_stored":60446735,"data_compressed":3110207,"data_compressed_allocated":29876224,"data_compressed_original":60243968,"omap_allocated":0,"internal_metadata":101384192},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":29913088,"data_stored":59826176,"data_compressed":3088291,"data_compressed_allocated":29913088,"data_compressed_original":59826176,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32477184,"data_stored":64954368,"data_compressed":3352953,"data_compressed_allocated":32477184,"data_compressed_original":64954368,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20713472,"data_stored":41426944,"data_compressed":2138523,"data_compressed_allocated":20713472,"data_compressed_original":41426944,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28213248,"data_stored":56426496,"data_compressed":2912774,"data_compressed_allocated":28213248,"data_compressed_original":56426496,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:55:25.525 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 69 16 9576 9560 1.08233 1.05469 0.146416 0.11538 2024-04-23T01:55:26.525 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 70 16 9705 9689 1.08126 1.00781 0.047933 0.115452 2024-04-23T01:55:27.525 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 71 16 9821 9805 1.0788 0.90625 0.0597155 0.115742 2024-04-23T01:55:27.609 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:55:27.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:27.726+0000 7f129d2ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:27.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:27.734+0000 7f129d2ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:28.525 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 72 16 9957 9941 1.07857 1.0625 0.0545012 0.115806 2024-04-23T01:55:29.525 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 73 16 10092 10076 1.07824 1.05469 0.0553849 0.115822 2024-04-23T01:55:30.001 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:55:30.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:30.152+0000 7f2d7826d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:30.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:30.159+0000 7f2d7826d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:30.442 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:55:30.442 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:55:30.442 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:55:30.442 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:55:30.525 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 74 16 10217 10201 1.07687 0.976562 0.0869703 0.115927 2024-04-23T01:55:30.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:30.566+0000 7f1132354640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:30.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:30.577+0000 7f1132354640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:30.708 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:55:30.709 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:55:30.709 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:55:30.709 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:55:30.709 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:55:30.709 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:55:30.709 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:55:30.709 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:55:30.709 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:55:30.709 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:55:30.709 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:55:30.709 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:55:30.712 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:55:30.712 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:55:30.712 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:55:30.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:30.834+0000 7f7f21417640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:30.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:30.845+0000 7f7f21417640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:30.991 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:55:30.991 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:55:30.991 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:55:30.991 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:55:30.991 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:55:30.991 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:55:30.991 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:55:30.991 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:55:30.992 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:55:30.992 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:55:30.992 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:55:30.992 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:55:30.994 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:55:30.995 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:55:30.995 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:55:31.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:31.140+0000 7fe4f42ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:31.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:31.153+0000 7fe4f42ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:31.171 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:55:31.171 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:55:31.171 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:55:31.171 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:55:31.171 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:55:31.172 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:55:31.172 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:55:31.172 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:55:31.172 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:55:31.172 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:55:31.172 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:55:31.172 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:55:31.175 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:55:31.176 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:55:31.176 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:55:31.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:31.290+0000 7fcf9699d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:31.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:31.303+0000 7fcf9699d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:31.320 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:55:31.320 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:55:31.320 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:55:31.320 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:55:31.320 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:55:31.320 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:55:31.320 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:55:31.321 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:55:31.321 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:55:31.321 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:55:31.321 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:55:31.321 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:55:31.323 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:55:31.324 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:55:31.525 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 75 16 10339 10323 1.07522 0.953125 0.122283 0.116122 2024-04-23T01:55:32.422 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:55:32.525 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 76 16 10470 10454 1.07453 1.02344 0.0743639 0.116152 2024-04-23T01:55:32.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:32.535+0000 7fa879cad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:32.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:32.543+0000 7fa879cad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:33.526 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 77 16 10604 10588 1.07417 1.04688 0.0612042 0.11625 2024-04-23T01:55:34.526 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 78 16 10733 10717 1.07332 1.00781 0.103092 0.116353 2024-04-23T01:55:34.734 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:55:34.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:34.869+0000 7f1255a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:34.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:34.876+0000 7f1255a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:35.526 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 79 16 10862 10846 1.07249 1.00781 0.194796 0.116426 2024-04-23T01:55:36.325 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:55:36.325 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:55:36.325 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:55:36.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:36.453+0000 7f8a6ba2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:36.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:36.463+0000 7f8a6ba2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:36.526 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:55:36.525852+0000 min lat: 0.0153179 max lat: 0.466783 avg lat: 0.116555 2024-04-23T01:55:36.526 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:55:36.526 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 80 16 10988 10972 1.07139 0.984375 0.191116 0.116555 2024-04-23T01:55:36.819 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:55:36.819 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:55:36.836 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-04-23T01:55:35.669453+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":85352992,"num_objects":1297,"num_object_clones":0,"num_object_copies":3891,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":10484,"num_write_kb":84670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10428,"ondisk_log_size":10428,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":537332,"kb_used_data":126036,"kb_used_omap":0,"kb_used_meta":411264,"kb_avail":376950028,"statfs":{"total":386547056640,"available":385996828672,"internally_reserved":0,"allocated":129060864,"data_stored":256138780,"data_compressed":13182609,"data_compressed_allocated":126935040,"data_compressed_original":255344640,"omap_allocated":0,"internal_metadata":421134336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10559488,"num_objects":161,"num_object_clones":0,"num_object_copies":483,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1289,"num_write_kb":10312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002430"},"pg_stats":[{"pgid":"8.7","version":"60'711","reported_seq":742,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:31.223680+0000","last_change":"2024-04-23T01:54:15.581616+0000","last_active":"2024-04-23T01:55:31.223680+0000","last_peered":"2024-04-23T01:55:31.223680+0000","last_clean":"2024-04-23T01:55:31.223680+0000","last_became_active":"2024-04-23T01:54:15.576738+0000","last_became_peered":"2024-04-23T01:54:15.576738+0000","last_unstale":"2024-04-23T01:55:31.223680+0000","last_undegraded":"2024-04-23T01:55:31.223680+0000","last_fullsized":"2024-04-23T01:55:31.223680+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5832704,"num_objects":89,"num_object_clones":0,"num_object_copies":267,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":712,"num_write_kb":5696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"60'655","reported_seq":686,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:31.674761+0000","last_change":"2024-04-23T01:54:15.596716+0000","last_active":"2024-04-23T01:55:31.674761+0000","last_peered":"2024-04-23T01:55:31.674761+0000","last_clean":"2024-04-23T01:55:31.674761+0000","last_became_active":"2024-04-23T01:54:15.589907+0000","last_became_peered":"2024-04-23T01:54:15.589907+0000","last_unstale":"2024-04-23T01:55:31.674761+0000","last_undegraded":"2024-04-23T01:55:31.674761+0000","last_fullsized":"2024-04-23T01:55:31.674761+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"60'543","reported_seq":576,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:30.940707+0000","last_change":"2024-04-23T01:54:15.700282+0000","last_active":"2024-04-23T01:55:30.940707+0000","last_peered":"2024-04-23T01:55:30.940707+0000","last_clean":"2024-04-23T01:55:30.940707+0000","last_became_active":"2024-04-23T01:54:15.686441+0000","last_became_peered":"2024-04-23T01:54:15.686441+0000","last_unstale":"2024-04-23T01:55:30.940707+0000","last_undegraded":"2024-04-23T01:55:30.940707+0000","last_fullsized":"2024-04-23T01:55:30.940707+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":544,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"60'727","reported_seq":758,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:29.540797+0000","last_change":"2024-04-23T01:54:15.681870+0000","last_active":"2024-04-23T01:55:29.540797+0000","last_peered":"2024-04-23T01:55:29.540797+0000","last_clean":"2024-04-23T01:55:29.540797+0000","last_became_active":"2024-04-23T01:54:15.677678+0000","last_became_peered":"2024-04-23T01:54:15.677678+0000","last_unstale":"2024-04-23T01:55:29.540797+0000","last_undegraded":"2024-04-23T01:55:29.540797+0000","last_fullsized":"2024-04-23T01:55:29.540797+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":728,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3","version":"60'639","reported_seq":672,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:31.398550+0000","last_change":"2024-04-23T01:54:15.711951+0000","last_active":"2024-04-23T01:55:31.398550+0000","last_peered":"2024-04-23T01:55:31.398550+0000","last_clean":"2024-04-23T01:55:31.398550+0000","last_became_active":"2024-04-23T01:54:15.696611+0000","last_became_peered":"2024-04-23T01:54:15.696611+0000","last_unstale":"2024-04-23T01:55:31.398550+0000","last_undegraded":"2024-04-23T01:55:31.398550+0000","last_fullsized":"2024-04-23T01:55:31.398550+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":640,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"60'679","reported_seq":710,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:30.583402+0000","last_change":"2024-04-23T01:54:15.676654+0000","last_active":"2024-04-23T01:55:30.583402+0000","last_peered":"2024-04-23T01:55:30.583402+0000","last_clean":"2024-04-23T01:55:30.583402+0000","last_became_active":"2024-04-23T01:54:15.672239+0000","last_became_peered":"2024-04-23T01:54:15.672239+0000","last_unstale":"2024-04-23T01:55:30.583402+0000","last_undegraded":"2024-04-23T01:55:30.583402+0000","last_fullsized":"2024-04-23T01:55:30.583402+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5570560,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":680,"num_write_kb":5440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"60'647","reported_seq":680,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:31.584671+0000","last_change":"2024-04-23T01:54:15.609601+0000","last_active":"2024-04-23T01:55:31.584671+0000","last_peered":"2024-04-23T01:55:31.584671+0000","last_clean":"2024-04-23T01:55:31.584671+0000","last_became_active":"2024-04-23T01:54:15.598424+0000","last_became_peered":"2024-04-23T01:54:15.598424+0000","last_unstale":"2024-04-23T01:55:31.584671+0000","last_undegraded":"2024-04-23T01:55:31.584671+0000","last_fullsized":"2024-04-23T01:55:31.584671+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308416,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":648,"num_write_kb":5184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"60'735","reported_seq":768,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:31.098743+0000","last_change":"2024-04-23T01:54:15.718344+0000","last_active":"2024-04-23T01:55:31.098743+0000","last_peered":"2024-04-23T01:55:31.098743+0000","last_clean":"2024-04-23T01:55:31.098743+0000","last_became_active":"2024-04-23T01:54:15.706071+0000","last_became_peered":"2024-04-23T01:54:15.706071+0000","last_unstale":"2024-04-23T01:55:31.098743+0000","last_undegraded":"2024-04-23T01:55:31.098743+0000","last_fullsized":"2024-04-23T01:55:31.098743+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6029312,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":736,"num_write_kb":5888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"60'591","reported_seq":622,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:31.261065+0000","last_change":"2024-04-23T01:54:15.596696+0000","last_active":"2024-04-23T01:55:31.261065+0000","last_peered":"2024-04-23T01:55:31.261065+0000","last_clean":"2024-04-23T01:55:31.261065+0000","last_became_active":"2024-04-23T01:54:15.588060+0000","last_became_peered":"2024-04-23T01:54:15.588060+0000","last_unstale":"2024-04-23T01:55:31.261065+0000","last_undegraded":"2024-04-23T01:55:31.261065+0000","last_fullsized":"2024-04-23T01:55:31.261065+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":592,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":267,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.696720+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:54:35.696720+0000","last_peered":"2024-04-23T01:54:35.696720+0000","last_clean":"2024-04-23T01:54:35.696720+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:54:35.696720+0000","last_undegraded":"2024-04-23T01:54:35.696720+0000","last_fullsized":"2024-04-23T01:54:35.696720+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":138,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.679304+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:54:35.679304+0000","last_peered":"2024-04-23T01:54:35.679304+0000","last_clean":"2024-04-23T01:54:35.679304+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:54:35.679304+0000","last_undegraded":"2024-04-23T01:54:35.679304+0000","last_fullsized":"2024-04-23T01:54:35.679304+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"60'551","reported_seq":584,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:29.805779+0000","last_change":"2024-04-23T01:54:15.705667+0000","last_active":"2024-04-23T01:55:29.805779+0000","last_peered":"2024-04-23T01:55:29.805779+0000","last_clean":"2024-04-23T01:55:29.805779+0000","last_became_active":"2024-04-23T01:54:15.701510+0000","last_became_peered":"2024-04-23T01:54:15.701510+0000","last_unstale":"2024-04-23T01:55:29.805779+0000","last_undegraded":"2024-04-23T01:55:29.805779+0000","last_fullsized":"2024-04-23T01:55:29.805779+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4521984,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":552,"num_write_kb":4416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":130,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.677386+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:54:35.677386+0000","last_peered":"2024-04-23T01:54:35.677386+0000","last_clean":"2024-04-23T01:54:35.677386+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:54:35.677386+0000","last_undegraded":"2024-04-23T01:54:35.677386+0000","last_fullsized":"2024-04-23T01:54:35.677386+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"60'655","reported_seq":686,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:31.399610+0000","last_change":"2024-04-23T01:54:15.668120+0000","last_active":"2024-04-23T01:55:31.399610+0000","last_peered":"2024-04-23T01:55:31.399610+0000","last_clean":"2024-04-23T01:55:31.399610+0000","last_became_active":"2024-04-23T01:54:15.661561+0000","last_became_peered":"2024-04-23T01:54:15.661561+0000","last_unstale":"2024-04-23T01:55:31.399610+0000","last_undegraded":"2024-04-23T01:55:31.399610+0000","last_fullsized":"2024-04-23T01:55:31.399610+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":138,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.657300+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:54:35.657300+0000","last_peered":"2024-04-23T01:54:35.657300+0000","last_clean":"2024-04-23T01:54:35.657300+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:54:35.657300+0000","last_undegraded":"2024-04-23T01:54:35.657300+0000","last_fullsized":"2024-04-23T01:54:35.657300+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"60'561","reported_seq":594,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:31.647111+0000","last_change":"2024-04-23T01:54:15.677434+0000","last_active":"2024-04-23T01:55:31.647111+0000","last_peered":"2024-04-23T01:55:31.647111+0000","last_clean":"2024-04-23T01:55:31.647111+0000","last_became_active":"2024-04-23T01:54:15.670406+0000","last_became_peered":"2024-04-23T01:54:15.670406+0000","last_unstale":"2024-04-23T01:55:31.647111+0000","last_undegraded":"2024-04-23T01:55:31.647111+0000","last_fullsized":"2024-04-23T01:55:31.647111+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4603904,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":562,"num_write_kb":4496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":130,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.679268+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:54:35.679268+0000","last_peered":"2024-04-23T01:54:35.679268+0000","last_clean":"2024-04-23T01:54:35.679268+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:54:35.679268+0000","last_undegraded":"2024-04-23T01:54:35.679268+0000","last_fullsized":"2024-04-23T01:54:35.679268+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.e","version":"60'638","reported_seq":669,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:31.620768+0000","last_change":"2024-04-23T01:54:15.652153+0000","last_active":"2024-04-23T01:55:31.620768+0000","last_peered":"2024-04-23T01:55:31.620768+0000","last_clean":"2024-04-23T01:55:31.620768+0000","last_became_active":"2024-04-23T01:54:15.644332+0000","last_became_peered":"2024-04-23T01:54:15.644332+0000","last_unstale":"2024-04-23T01:55:31.620768+0000","last_undegraded":"2024-04-23T01:55:31.620768+0000","last_fullsized":"2024-04-23T01:55:31.620768+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5234688,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":639,"num_write_kb":5112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":136,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:33.277036+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:54:33.277036+0000","last_peered":"2024-04-23T01:54:33.277036+0000","last_clean":"2024-04-23T01:54:33.277036+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:54:33.277036+0000","last_undegraded":"2024-04-23T01:54:33.277036+0000","last_fullsized":"2024-04-23T01:54:33.277036+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"60'711","reported_seq":744,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:28.614363+0000","last_change":"2024-04-23T01:54:15.637842+0000","last_active":"2024-04-23T01:55:28.614363+0000","last_peered":"2024-04-23T01:55:28.614363+0000","last_clean":"2024-04-23T01:55:28.614363+0000","last_became_active":"2024-04-23T01:54:15.631530+0000","last_became_peered":"2024-04-23T01:54:15.631530+0000","last_unstale":"2024-04-23T01:55:28.614363+0000","last_undegraded":"2024-04-23T01:55:28.614363+0000","last_fullsized":"2024-04-23T01:55:28.614363+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5832704,"num_objects":89,"num_object_clones":0,"num_object_copies":267,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":712,"num_write_kb":5696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":134,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.712297+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:54:35.712297+0000","last_peered":"2024-04-23T01:54:35.712297+0000","last_clean":"2024-04-23T01:54:35.712297+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:54:35.712297+0000","last_undegraded":"2024-04-23T01:54:35.712297+0000","last_fullsized":"2024-04-23T01:54:35.712297+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"60'633","reported_seq":664,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:31.600027+0000","last_change":"2024-04-23T01:54:15.617410+0000","last_active":"2024-04-23T01:55:31.600027+0000","last_peered":"2024-04-23T01:55:31.600027+0000","last_clean":"2024-04-23T01:55:31.600027+0000","last_became_active":"2024-04-23T01:54:15.610208+0000","last_became_peered":"2024-04-23T01:54:15.610208+0000","last_unstale":"2024-04-23T01:55:31.600027+0000","last_undegraded":"2024-04-23T01:55:31.600027+0000","last_fullsized":"2024-04-23T01:55:31.600027+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5193728,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":634,"num_write_kb":5072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":136,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:33.278089+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:54:33.278089+0000","last_peered":"2024-04-23T01:54:33.278089+0000","last_clean":"2024-04-23T01:54:33.278089+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:54:33.278089+0000","last_undegraded":"2024-04-23T01:54:33.278089+0000","last_fullsized":"2024-04-23T01:54:33.278089+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"60'655","reported_seq":686,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:31.166699+0000","last_change":"2024-04-23T01:54:15.630052+0000","last_active":"2024-04-23T01:55:31.166699+0000","last_peered":"2024-04-23T01:55:31.166699+0000","last_clean":"2024-04-23T01:55:31.166699+0000","last_became_active":"2024-04-23T01:54:15.624615+0000","last_became_peered":"2024-04-23T01:54:15.624615+0000","last_unstale":"2024-04-23T01:55:31.166699+0000","last_undegraded":"2024-04-23T01:55:31.166699+0000","last_fullsized":"2024-04-23T01:55:31.166699+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":138,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.658618+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:54:35.658618+0000","last_peered":"2024-04-23T01:54:35.658618+0000","last_clean":"2024-04-23T01:54:35.658618+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:54:35.658618+0000","last_undegraded":"2024-04-23T01:54:35.658618+0000","last_fullsized":"2024-04-23T01:54:35.658618+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":84762624,"num_objects":1295,"num_object_clones":0,"num_object_copies":3885,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10347,"num_write_kb":82776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":127143936,"data_stored":254287872,"data_compressed":13127013,"data_compressed_allocated":127143936,"data_compressed_original":254287872,"omap_allocated":0,"internal_metadata":0},"log_size":10331,"ondisk_log_size":10331,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706288,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":134980,"kb_used_data":31804,"kb_used_omap":0,"kb_used_meta":103168,"kb_avail":94236860,"statfs":{"total":96636764160,"available":96498544640,"internally_reserved":0,"allocated":32567296,"data_stored":64755727,"data_compressed":3332733,"data_compressed_allocated":32030720,"data_compressed_original":64552960,"omap_allocated":0,"internal_metadata":105644032},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706288,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":117004,"kb_used_data":23556,"kb_used_omap":0,"kb_used_meta":93440,"kb_avail":94254836,"statfs":{"total":96636764160,"available":96516952064,"internally_reserved":0,"allocated":24121344,"data_stored":47863823,"data_compressed":2460769,"data_compressed_allocated":23584768,"data_compressed_original":47661056,"omap_allocated":0,"internal_metadata":95682560},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706288,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":148204,"kb_used_data":36708,"kb_used_omap":0,"kb_used_meta":111488,"kb_avail":94223636,"statfs":{"total":96636764160,"available":96485003264,"internally_reserved":0,"allocated":37588992,"data_stored":74331631,"data_compressed":3827559,"data_compressed_allocated":37072896,"data_compressed_original":74145792,"omap_allocated":0,"internal_metadata":114163712},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706288,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":137144,"kb_used_data":33968,"kb_used_omap":0,"kb_used_meta":103168,"kb_avail":94234696,"statfs":{"total":96636764160,"available":96496328704,"internally_reserved":0,"allocated":34783232,"data_stored":69187599,"data_compressed":3561548,"data_compressed_allocated":34246656,"data_compressed_original":68984832,"omap_allocated":0,"internal_metadata":105644032},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":34275328,"data_stored":68550656,"data_compressed":3538786,"data_compressed_allocated":34275328,"data_compressed_original":68550656,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":37167104,"data_stored":74334208,"data_compressed":3837288,"data_compressed_allocated":37167104,"data_compressed_original":74334208,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":23592960,"data_stored":47185920,"data_compressed":2435892,"data_compressed_allocated":23592960,"data_compressed_original":47185920,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32108544,"data_stored":64217088,"data_compressed":3315047,"data_compressed_allocated":32108544,"data_compressed_original":64217088,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:55:37.130 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:55:37.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:37.249+0000 7f5551a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:37.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:37.257+0000 7f5551a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:37.526 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 81 16 11092 11076 1.06819 0.8125 0.102022 0.116884 2024-04-23T01:55:38.526 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 82 16 11220 11204 1.06736 1 0.143483 0.117021 2024-04-23T01:55:39.501 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:55:39.526 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 83 16 11350 11334 1.06673 1.01562 0.142447 0.117072 2024-04-23T01:55:39.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:39.628+0000 7fc49b06d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:39.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:39.635+0000 7fc49b06d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:40.526 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 84 16 11466 11450 1.06482 0.90625 0.171357 0.117266 2024-04-23T01:55:41.526 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 85 16 11606 11590 1.06516 1.09375 0.114195 0.117237 2024-04-23T01:55:41.770 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:55:41.838 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:55:41.839 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:55:41.839 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:55:41.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:41.898+0000 7f084826d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:41.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:41.905+0000 7f084826d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:41.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:41.964+0000 7fdcc4b5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:41.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:41.976+0000 7fdcc4b5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:42.194 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:55:42.195 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:55:42.209 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2024-04-23T01:55:41.670601+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":92856864,"num_objects":1410,"num_object_clones":0,"num_object_copies":4230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":11400,"num_write_kb":91998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11344,"ondisk_log_size":11344,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":555460,"kb_used_data":135840,"kb_used_omap":0,"kb_used_meta":419584,"kb_avail":376931900,"statfs":{"total":386547056640,"available":385978265600,"internally_reserved":0,"allocated":139100160,"data_stored":276217372,"data_compressed":14219381,"data_compressed_allocated":136974336,"data_compressed_original":275423232,"omap_allocated":0,"internal_metadata":429654016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12763136,"num_objects":194,"num_object_clones":0,"num_object_copies":582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1558,"num_write_kb":12464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002385"},"pg_stats":[{"pgid":"8.7","version":"60'751","reported_seq":782,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:35.780682+0000","last_change":"2024-04-23T01:54:15.581616+0000","last_active":"2024-04-23T01:55:35.780682+0000","last_peered":"2024-04-23T01:55:35.780682+0000","last_clean":"2024-04-23T01:55:35.780682+0000","last_became_active":"2024-04-23T01:54:15.576738+0000","last_became_peered":"2024-04-23T01:54:15.576738+0000","last_unstale":"2024-04-23T01:55:35.780682+0000","last_undegraded":"2024-04-23T01:55:35.780682+0000","last_fullsized":"2024-04-23T01:55:35.780682+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"60'695","reported_seq":726,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:34.464536+0000","last_change":"2024-04-23T01:54:15.596716+0000","last_active":"2024-04-23T01:55:34.464536+0000","last_peered":"2024-04-23T01:55:34.464536+0000","last_clean":"2024-04-23T01:55:34.464536+0000","last_became_active":"2024-04-23T01:54:15.589907+0000","last_became_peered":"2024-04-23T01:54:15.589907+0000","last_unstale":"2024-04-23T01:55:34.464536+0000","last_undegraded":"2024-04-23T01:55:34.464536+0000","last_fullsized":"2024-04-23T01:55:34.464536+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":696,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"60'559","reported_seq":592,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:34.960457+0000","last_change":"2024-04-23T01:54:15.700282+0000","last_active":"2024-04-23T01:55:34.960457+0000","last_peered":"2024-04-23T01:55:34.960457+0000","last_clean":"2024-04-23T01:55:34.960457+0000","last_became_active":"2024-04-23T01:54:15.686441+0000","last_became_peered":"2024-04-23T01:54:15.686441+0000","last_unstale":"2024-04-23T01:55:34.960457+0000","last_undegraded":"2024-04-23T01:55:34.960457+0000","last_fullsized":"2024-04-23T01:55:34.960457+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4587520,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":560,"num_write_kb":4480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"60'799","reported_seq":830,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:41.415170+0000","last_change":"2024-04-23T01:54:15.681870+0000","last_active":"2024-04-23T01:55:41.415170+0000","last_peered":"2024-04-23T01:55:41.415170+0000","last_clean":"2024-04-23T01:55:41.415170+0000","last_became_active":"2024-04-23T01:54:15.677678+0000","last_became_peered":"2024-04-23T01:54:15.677678+0000","last_unstale":"2024-04-23T01:55:41.415170+0000","last_undegraded":"2024-04-23T01:55:41.415170+0000","last_fullsized":"2024-04-23T01:55:41.415170+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":799,"log_dups_size":0,"ondisk_log_size":799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6553600,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":800,"num_write_kb":6400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3","version":"60'687","reported_seq":720,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:36.138657+0000","last_change":"2024-04-23T01:54:15.711951+0000","last_active":"2024-04-23T01:55:36.138657+0000","last_peered":"2024-04-23T01:55:36.138657+0000","last_clean":"2024-04-23T01:55:36.138657+0000","last_became_active":"2024-04-23T01:54:15.696611+0000","last_became_peered":"2024-04-23T01:54:15.696611+0000","last_unstale":"2024-04-23T01:55:36.138657+0000","last_undegraded":"2024-04-23T01:55:36.138657+0000","last_fullsized":"2024-04-23T01:55:36.138657+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5636096,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":688,"num_write_kb":5504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"60'759","reported_seq":790,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:41.550084+0000","last_change":"2024-04-23T01:54:15.676654+0000","last_active":"2024-04-23T01:55:41.550084+0000","last_peered":"2024-04-23T01:55:41.550084+0000","last_clean":"2024-04-23T01:55:41.550084+0000","last_became_active":"2024-04-23T01:54:15.672239+0000","last_became_peered":"2024-04-23T01:54:15.672239+0000","last_unstale":"2024-04-23T01:55:41.550084+0000","last_undegraded":"2024-04-23T01:55:41.550084+0000","last_fullsized":"2024-04-23T01:55:41.550084+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6225920,"num_objects":95,"num_object_clones":0,"num_object_copies":285,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":760,"num_write_kb":6080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"60'663","reported_seq":696,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:35.996627+0000","last_change":"2024-04-23T01:54:15.609601+0000","last_active":"2024-04-23T01:55:35.996627+0000","last_peered":"2024-04-23T01:55:35.996627+0000","last_clean":"2024-04-23T01:55:35.996627+0000","last_became_active":"2024-04-23T01:54:15.598424+0000","last_became_peered":"2024-04-23T01:54:15.598424+0000","last_unstale":"2024-04-23T01:55:35.996627+0000","last_undegraded":"2024-04-23T01:55:35.996627+0000","last_fullsized":"2024-04-23T01:55:35.996627+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5439488,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":664,"num_write_kb":5312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"60'775","reported_seq":808,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:35.342769+0000","last_change":"2024-04-23T01:54:15.718344+0000","last_active":"2024-04-23T01:55:35.342769+0000","last_peered":"2024-04-23T01:55:35.342769+0000","last_clean":"2024-04-23T01:55:35.342769+0000","last_became_active":"2024-04-23T01:54:15.706071+0000","last_became_peered":"2024-04-23T01:54:15.706071+0000","last_unstale":"2024-04-23T01:55:35.342769+0000","last_undegraded":"2024-04-23T01:55:35.342769+0000","last_fullsized":"2024-04-23T01:55:35.342769+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6356992,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":776,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"60'679","reported_seq":710,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:41.298581+0000","last_change":"2024-04-23T01:54:15.596696+0000","last_active":"2024-04-23T01:55:41.298581+0000","last_peered":"2024-04-23T01:55:41.298581+0000","last_clean":"2024-04-23T01:55:41.298581+0000","last_became_active":"2024-04-23T01:54:15.588060+0000","last_became_peered":"2024-04-23T01:54:15.588060+0000","last_unstale":"2024-04-23T01:55:41.298581+0000","last_undegraded":"2024-04-23T01:55:41.298581+0000","last_fullsized":"2024-04-23T01:55:41.298581+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5570560,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":680,"num_write_kb":5440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":267,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.696720+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:54:35.696720+0000","last_peered":"2024-04-23T01:54:35.696720+0000","last_clean":"2024-04-23T01:54:35.696720+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:54:35.696720+0000","last_undegraded":"2024-04-23T01:54:35.696720+0000","last_fullsized":"2024-04-23T01:54:35.696720+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":138,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.679304+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:54:35.679304+0000","last_peered":"2024-04-23T01:54:35.679304+0000","last_clean":"2024-04-23T01:54:35.679304+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:54:35.679304+0000","last_undegraded":"2024-04-23T01:54:35.679304+0000","last_fullsized":"2024-04-23T01:54:35.679304+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"60'623","reported_seq":656,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:36.676096+0000","last_change":"2024-04-23T01:54:15.705667+0000","last_active":"2024-04-23T01:55:36.676096+0000","last_peered":"2024-04-23T01:55:36.676096+0000","last_clean":"2024-04-23T01:55:36.676096+0000","last_became_active":"2024-04-23T01:54:15.701510+0000","last_became_peered":"2024-04-23T01:54:15.701510+0000","last_unstale":"2024-04-23T01:55:36.676096+0000","last_undegraded":"2024-04-23T01:55:36.676096+0000","last_fullsized":"2024-04-23T01:55:36.676096+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5111808,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":624,"num_write_kb":4992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":130,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.677386+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:54:35.677386+0000","last_peered":"2024-04-23T01:54:35.677386+0000","last_clean":"2024-04-23T01:54:35.677386+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:54:35.677386+0000","last_undegraded":"2024-04-23T01:54:35.677386+0000","last_fullsized":"2024-04-23T01:54:35.677386+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"60'751","reported_seq":782,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:40.907474+0000","last_change":"2024-04-23T01:54:15.668120+0000","last_active":"2024-04-23T01:55:40.907474+0000","last_peered":"2024-04-23T01:55:40.907474+0000","last_clean":"2024-04-23T01:55:40.907474+0000","last_became_active":"2024-04-23T01:54:15.661561+0000","last_became_peered":"2024-04-23T01:54:15.661561+0000","last_unstale":"2024-04-23T01:55:40.907474+0000","last_undegraded":"2024-04-23T01:55:40.907474+0000","last_fullsized":"2024-04-23T01:55:40.907474+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":138,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.657300+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:54:35.657300+0000","last_peered":"2024-04-23T01:54:35.657300+0000","last_clean":"2024-04-23T01:54:35.657300+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:54:35.657300+0000","last_undegraded":"2024-04-23T01:54:35.657300+0000","last_fullsized":"2024-04-23T01:54:35.657300+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"60'575","reported_seq":608,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:34.296331+0000","last_change":"2024-04-23T01:54:15.677434+0000","last_active":"2024-04-23T01:55:34.296331+0000","last_peered":"2024-04-23T01:55:34.296331+0000","last_clean":"2024-04-23T01:55:34.296331+0000","last_became_active":"2024-04-23T01:54:15.670406+0000","last_became_peered":"2024-04-23T01:54:15.670406+0000","last_unstale":"2024-04-23T01:55:34.296331+0000","last_undegraded":"2024-04-23T01:55:34.296331+0000","last_fullsized":"2024-04-23T01:55:34.296331+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718592,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":576,"num_write_kb":4608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":130,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.679268+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:54:35.679268+0000","last_peered":"2024-04-23T01:54:35.679268+0000","last_clean":"2024-04-23T01:54:35.679268+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:54:35.679268+0000","last_undegraded":"2024-04-23T01:54:35.679268+0000","last_fullsized":"2024-04-23T01:54:35.679268+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.e","version":"60'742","reported_seq":773,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:41.612455+0000","last_change":"2024-04-23T01:54:15.652153+0000","last_active":"2024-04-23T01:55:41.612455+0000","last_peered":"2024-04-23T01:55:41.612455+0000","last_clean":"2024-04-23T01:55:41.612455+0000","last_became_active":"2024-04-23T01:54:15.644332+0000","last_became_peered":"2024-04-23T01:54:15.644332+0000","last_unstale":"2024-04-23T01:55:41.612455+0000","last_undegraded":"2024-04-23T01:55:41.612455+0000","last_fullsized":"2024-04-23T01:55:41.612455+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6086656,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":743,"num_write_kb":5944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":136,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:33.277036+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:54:33.277036+0000","last_peered":"2024-04-23T01:54:33.277036+0000","last_clean":"2024-04-23T01:54:33.277036+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:54:33.277036+0000","last_undegraded":"2024-04-23T01:54:33.277036+0000","last_fullsized":"2024-04-23T01:54:33.277036+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"60'767","reported_seq":800,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:35.676786+0000","last_change":"2024-04-23T01:54:15.637842+0000","last_active":"2024-04-23T01:55:35.676786+0000","last_peered":"2024-04-23T01:55:35.676786+0000","last_clean":"2024-04-23T01:55:35.676786+0000","last_became_active":"2024-04-23T01:54:15.631530+0000","last_became_peered":"2024-04-23T01:54:15.631530+0000","last_unstale":"2024-04-23T01:55:35.676786+0000","last_undegraded":"2024-04-23T01:55:35.676786+0000","last_fullsized":"2024-04-23T01:55:35.676786+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":768,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":134,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.712297+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:54:35.712297+0000","last_peered":"2024-04-23T01:54:35.712297+0000","last_clean":"2024-04-23T01:54:35.712297+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:54:35.712297+0000","last_undegraded":"2024-04-23T01:54:35.712297+0000","last_fullsized":"2024-04-23T01:54:35.712297+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"60'703","reported_seq":734,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:41.378889+0000","last_change":"2024-04-23T01:54:15.617410+0000","last_active":"2024-04-23T01:55:41.378889+0000","last_peered":"2024-04-23T01:55:41.378889+0000","last_clean":"2024-04-23T01:55:41.378889+0000","last_became_active":"2024-04-23T01:54:15.610208+0000","last_became_peered":"2024-04-23T01:54:15.610208+0000","last_unstale":"2024-04-23T01:55:41.378889+0000","last_undegraded":"2024-04-23T01:55:41.378889+0000","last_fullsized":"2024-04-23T01:55:41.378889+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767168,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":704,"num_write_kb":5632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":136,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:33.278089+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:54:33.278089+0000","last_peered":"2024-04-23T01:54:33.278089+0000","last_clean":"2024-04-23T01:54:33.278089+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:54:33.278089+0000","last_undegraded":"2024-04-23T01:54:33.278089+0000","last_fullsized":"2024-04-23T01:54:33.278089+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"60'719","reported_seq":750,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:55:39.752460+0000","last_change":"2024-04-23T01:54:15.630052+0000","last_active":"2024-04-23T01:55:39.752460+0000","last_peered":"2024-04-23T01:55:39.752460+0000","last_clean":"2024-04-23T01:55:39.752460+0000","last_became_active":"2024-04-23T01:54:15.624615+0000","last_became_peered":"2024-04-23T01:54:15.624615+0000","last_unstale":"2024-04-23T01:55:39.752460+0000","last_undegraded":"2024-04-23T01:55:39.752460+0000","last_fullsized":"2024-04-23T01:55:39.752460+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:54:14.421067+0000","last_clean_scrub_stamp":"2024-04-23T01:54:14.421067+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":720,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":138,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-23T01:54:35.658618+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:54:35.658618+0000","last_peered":"2024-04-23T01:54:35.658618+0000","last_clean":"2024-04-23T01:54:35.658618+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:54:35.658618+0000","last_undegraded":"2024-04-23T01:54:35.658618+0000","last_fullsized":"2024-04-23T01:54:35.658618+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":92266496,"num_objects":1408,"num_object_clones":0,"num_object_copies":4224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11263,"num_write_kb":90104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":137228288,"data_stored":274456576,"data_compressed":14168438,"data_compressed_allocated":137228288,"data_compressed_original":274456576,"omap_allocated":0,"internal_metadata":0},"log_size":11247,"ondisk_log_size":11247,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706293,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":136964,"kb_used_data":33788,"kb_used_omap":0,"kb_used_meta":103168,"kb_avail":94234876,"statfs":{"total":96636764160,"available":96496513024,"internally_reserved":0,"allocated":34598912,"data_stored":68818959,"data_compressed":3542541,"data_compressed_allocated":34062336,"data_compressed_original":68616192,"omap_allocated":0,"internal_metadata":105644032},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706293,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":122748,"kb_used_data":25140,"kb_used_omap":0,"kb_used_meta":97600,"kb_avail":94249092,"statfs":{"total":96636764160,"available":96511070208,"internally_reserved":0,"allocated":25743360,"data_stored":51107855,"data_compressed":2628277,"data_compressed_allocated":25206784,"data_compressed_original":50905088,"omap_allocated":0,"internal_metadata":99942400},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706293,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":150412,"kb_used_data":38916,"kb_used_omap":0,"kb_used_meta":111488,"kb_avail":94221428,"statfs":{"total":96636764160,"available":96482742272,"internally_reserved":0,"allocated":39849984,"data_stored":78853615,"data_compressed":4061055,"data_compressed_allocated":39333888,"data_compressed_original":78667776,"omap_allocated":0,"internal_metadata":114163712},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706298,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":145336,"kb_used_data":37996,"kb_used_omap":0,"kb_used_meta":107328,"kb_avail":94226504,"statfs":{"total":96636764160,"available":96487940096,"internally_reserved":0,"allocated":38907904,"data_stored":77436943,"data_compressed":3987508,"data_compressed_allocated":38371328,"data_compressed_original":77234176,"omap_allocated":0,"internal_metadata":109903872},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":38379520,"data_stored":76759040,"data_compressed":3962631,"data_compressed_allocated":38379520,"data_compressed_original":76759040,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":39419904,"data_stored":78839808,"data_compressed":4069938,"data_compressed_allocated":39419904,"data_compressed_original":78839808,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":25317376,"data_stored":50634752,"data_compressed":2613975,"data_compressed_allocated":25317376,"data_compressed_original":50634752,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":34111488,"data_stored":68222976,"data_compressed":3521894,"data_compressed_allocated":34111488,"data_compressed_original":68222976,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:55:42.526 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 86 16 11719 11703 1.06304 0.882812 0.131648 0.117505 2024-04-23T01:55:43.526 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 87 16 11847 11831 1.06231 1 0.0335026 0.117534 2024-04-23T01:55:44.121 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:55:44.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:44.242+0000 7fbf4b22d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:44.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:44.249+0000 7fbf4b22d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:44.526 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 88 16 11965 11949 1.06072 0.921875 0.177147 0.117747 2024-04-23T01:55:45.527 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 89 16 12098 12082 1.06047 1.03906 0.198086 0.117727 2024-04-23T01:55:46.421 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:55:46.527 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 90 16 12225 12209 1.05971 0.992188 0.147686 0.117847 2024-04-23T01:55:46.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:46.546+0000 7f3976b5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:46.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:46.553+0000 7f3976b5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:47.212 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:55:47.212 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:55:47.212 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:55:47.212 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:55:47.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:47.315+0000 7f91e035d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:47.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:47.324+0000 7f91e035d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:47.341 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:55:47.341 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:55:47.341 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:55:47.341 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:55:47.341 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:55:47.341 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:55:47.341 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:55:47.342 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:55:47.342 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:55:47.342 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:55:47.342 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:55:47.342 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:55:47.344 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:55:47.344 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:55:47.345 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:55:47.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:47.445+0000 7f78e100f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:47.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:47.453+0000 7f78e100f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:47.470 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:55:47.471 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:55:47.471 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:55:47.471 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:55:47.471 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:55:47.471 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:55:47.471 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:55:47.471 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:55:47.471 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:55:47.471 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:55:47.471 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:55:47.471 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:55:47.473 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:55:47.474 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:55:47.474 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:55:47.527 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Total time run: 90.1839 2024-04-23T01:55:47.527 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Total writes made: 12225 2024-04-23T01:55:47.527 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Write size: 8192 2024-04-23T01:55:47.527 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Object size: 65536 2024-04-23T01:55:47.527 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Bandwidth (MB/sec): 1.05903 2024-04-23T01:55:47.527 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Stddev Bandwidth: 0.0884749 2024-04-23T01:55:47.528 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Max bandwidth (MB/sec): 1.28906 2024-04-23T01:55:47.528 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Min bandwidth (MB/sec): 0.8125 2024-04-23T01:55:47.528 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Average IOPS: 135 2024-04-23T01:55:47.528 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Stddev IOPS: 11.3248 2024-04-23T01:55:47.528 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Max IOPS: 165 2024-04-23T01:55:47.528 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Min IOPS: 104 2024-04-23T01:55:47.528 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Average Latency(s): 0.117885 2024-04-23T01:55:47.528 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Stddev Latency(s): 0.05849 2024-04-23T01:55:47.528 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Max latency(s): 0.466783 2024-04-23T01:55:47.528 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Min latency(s): 0.0153179 2024-04-23T01:55:47.560 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Cleaning up (deleting benchmark objects) 2024-04-23T01:55:47.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:47.597+0000 7fd2a6024640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:47.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:47.608+0000 7fd2a6024640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:47.627 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:55:47.627 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:55:47.627 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:55:47.627 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:55:47.627 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:55:47.627 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:55:47.627 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:55:47.628 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:55:47.628 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:55:47.628 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:55:47.628 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:55:47.628 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:55:47.630 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:55:47.631 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:55:47.631 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:55:47.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:47.758+0000 7f3349ead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:47.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:47.770+0000 7f3349ead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:47.793 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:55:47.793 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:55:47.793 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:55:47.793 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:55:47.793 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:55:47.793 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:55:47.794 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:55:47.794 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:55:47.794 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:55:47.794 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:55:47.794 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:55:47.794 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:55:47.796 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:55:47.796 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:55:48.694 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:55:48.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:48.824+0000 7f746222d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:48.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:48.833+0000 7f746222d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:51.204 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:55:51.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:51.333+0000 7f12e502d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:51.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:51.340+0000 7f12e502d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:52.797 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:55:52.797 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:55:52.798 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:55:52.798 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:55:52.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:52.944+0000 7fdb93b9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:52.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:52.959+0000 7fdb93b9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:53.077 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:55:53.077 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:55:53.077 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:55:53.077 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:55:53.077 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:55:53.077 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:55:53.077 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:55:53.077 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:55:53.077 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:55:53.078 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:55:53.078 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:55:53.078 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:55:53.081 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:55:53.081 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:55:53.082 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:55:53.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:53.206+0000 7f71a4e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:53.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:53.222+0000 7f71a4e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:53.383 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:55:53.384 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:55:53.384 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:55:53.384 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:55:53.384 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:55:53.384 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:55:53.384 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:55:53.384 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:55:53.384 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:55:53.384 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:55:53.384 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:55:53.384 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:55:53.386 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:55:53.387 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:55:53.387 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:55:53.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:53.504+0000 7fdd0d4ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:53.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:53.530+0000 7fdd0d4ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:53.547 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:55:53.547 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:55:53.547 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:55:53.547 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:55:53.547 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:55:53.547 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:55:53.547 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:55:53.547 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:55:53.547 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:55:53.547 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:55:53.547 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:55:53.547 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:55:53.550 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:55:53.550 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:55:53.551 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:55:53.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:53.672+0000 7f0f78f5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:53.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:53.684+0000 7f0f78f5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:53.728 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:55:53.728 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:55:53.728 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:55:53.728 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:55:53.728 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:55:53.728 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:55:53.728 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:55:53.728 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:55:53.729 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:55:53.729 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:55:53.729 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:55:53.729 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:55:53.731 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:55:53.731 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:55:53.766 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:55:53.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:53.904+0000 7f3e2755d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:53.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:53.912+0000 7f3e2755d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:56.277 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:55:56.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:56.411+0000 7f09e04ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:56.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:56.418+0000 7f09e04ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:58.202 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Removed 1529 objects 2024-04-23T01:55:58.202 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Clean up completed and total clean up time :10.6423 2024-04-23T01:55:58.732 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:55:58.732 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:55:58.732 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:55:58.732 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:55:58.825 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:55:58.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:58.841+0000 7f6041ead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:58.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:58.851+0000 7f6041ead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:58.873 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:55:58.873 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:55:58.873 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:55:58.873 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:55:58.873 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:55:58.873 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:55:58.873 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:55:58.873 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:55:58.873 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:55:58.874 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:55:58.874 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:55:58.874 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:55:58.876 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:55:58.877 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:55:58.877 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:55:58.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:58.933+0000 7f5f17ead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:58.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:58.940+0000 7f5f17ead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:59.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:59.033+0000 7f76fec6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:59.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:59.043+0000 7f76fec6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:59.062 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:55:59.063 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:55:59.063 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:55:59.063 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:55:59.063 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:55:59.063 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:55:59.063 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:55:59.063 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:55:59.063 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:55:59.063 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:55:59.063 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:55:59.063 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:55:59.065 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:55:59.065 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:55:59.066 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:55:59.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:59.206+0000 7f7c5e46d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:59.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:59.214+0000 7f7c5e46d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:59.230 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:55:59.231 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:55:59.231 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:55:59.231 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:55:59.231 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:55:59.231 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:55:59.231 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:55:59.231 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:55:59.231 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:55:59.231 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:55:59.231 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:55:59.231 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:55:59.233 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:55:59.234 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:55:59.234 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:55:59.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:59.335+0000 7f6e4dced640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:59.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:55:59.344+0000 7f6e4dced640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:55:59.362 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:55:59.362 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:55:59.362 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:55:59.362 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:55:59.362 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:55:59.362 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:55:59.362 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:55:59.362 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:55:59.363 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:55:59.363 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:55:59.363 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:55:59.363 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:55:59.365 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:55:59.365 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:56:00.179 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_5 2024-04-23T01:56:00.180 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_5 unique_pool_5 --yes-i-really-really-mean-it 2024-04-23T01:56:00.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:00.281+0000 7fb761a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:00.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:00.293+0000 7fb761a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:01.053 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:56:01.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:01.154+0000 7f5e4882d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:01.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:01.159+0000 7f5e4882d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:01.362 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'unique_pool_5' removed 2024-04-23T01:56:01.377 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-04-23T01:56:01.378 INFO:tasks.radosbench:Beginning radosbench... 2024-04-23T01:56:01.378 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_6 2024-04-23T01:56:01.379 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_6 16 2024-04-23T01:56:01.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:01.480+0000 7f545806d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:01.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:01.491+0000 7f545806d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:02.370 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'unique_pool_6' created 2024-04-23T01:56:02.393 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_6 rados --yes-i-really-mean-it || true 2024-04-23T01:56:02.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:02.557+0000 7fb26535d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:02.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:02.573+0000 7fb26535d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:03.257 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:56:03.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:03.362+0000 7f6ed98ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:03.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:03.368+0000 7f6ed98ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:03.381 INFO:teuthology.orchestra.run.smithi090.stderr:enabled application 'rados' on pool 'unique_pool_6' 2024-04-23T01:56:04.366 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:56:04.366 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:56:04.366 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:56:04.367 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:56:04.402 DEBUG:teuthology.orchestra.run.smithi090:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 8192 --object-size 65536 -p unique_pool_6 bench 90 write' 2024-04-23T01:56:04.412 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-04-23T01:56:04.412 INFO:teuthology.orchestra.run:waiting for 3000 2024-04-23T01:56:04.475 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:hints = 1 2024-04-23T01:56:04.475 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Maintaining 16 concurrent writes of 8192 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-04-23T01:56:04.475 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Object prefix: benchmark_data_smithi090_55247 2024-04-23T01:56:04.475 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:56:04.475 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 0 0 0 0 0 0 - 0 2024-04-23T01:56:04.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:04.473+0000 7f721b26d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:04.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:04.484+0000 7f721b26d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:04.525 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:56:04.526 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:56:04.526 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:56:04.526 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:56:04.526 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:56:04.526 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:56:04.526 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:56:04.526 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:56:04.526 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:56:04.526 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:56:04.526 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:56:04.526 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:56:04.528 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:56:04.529 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:56:04.529 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:56:04.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:04.685+0000 7fcf67cad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:04.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:04.696+0000 7fcf67cad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:04.718 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:56:04.718 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:56:04.719 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:56:04.719 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:56:04.719 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:56:04.719 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:56:04.719 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:56:04.719 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:56:04.719 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:56:04.719 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:56:04.719 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:56:04.719 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:56:04.721 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:56:04.722 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:56:04.722 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:56:04.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:04.846+0000 7fefa855d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:04.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:04.855+0000 7fefa855d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:04.874 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:56:04.874 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:56:04.874 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:56:04.874 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:56:04.874 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:56:04.874 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:56:04.875 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:56:04.875 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:56:04.875 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:56:04.875 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:56:04.875 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:56:04.875 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:56:04.877 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:56:04.878 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:56:04.878 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:56:05.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:05.001+0000 7f4f1439d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:05.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:05.014+0000 7f4f1439d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:05.082 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:56:05.083 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:56:05.083 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:56:05.083 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:56:05.083 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:56:05.083 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:56:05.083 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:56:05.083 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:56:05.083 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:56:05.083 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:56:05.083 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:56:05.083 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:56:05.085 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:56:05.086 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:56:05.475 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 1 16 144 128 0.999944 1 0.215271 0.119332 2024-04-23T01:56:05.712 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:56:05.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:05.835+0000 7fb63d75d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:05.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:05.841+0000 7fb63d75d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:06.475 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 2 16 293 277 1.08196 1.16406 0.158466 0.112293 2024-04-23T01:56:07.475 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 3 16 445 429 1.11711 1.1875 0.171685 0.109334 2024-04-23T01:56:08.134 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:56:08.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:08.259+0000 7fa0feaad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:08.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:08.267+0000 7fa0feaad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:08.478 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 4 16 597 581 1.13468 1.1875 0.138613 0.108576 2024-04-23T01:56:09.475 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 5 16 768 752 1.1749 1.33594 0.13317 0.105412 2024-04-23T01:56:10.087 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:56:10.087 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-23T01:56:10.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:10.201+0000 7f023a6ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:10.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:10.215+0000 7f023a6ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:10.464 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:56:10.475 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 6 16 922 906 1.17959 1.20312 0.107566 0.104643 2024-04-23T01:56:10.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:10.602+0000 7f6e27d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:10.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:10.609+0000 7f6e27d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:11.393 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.0 to 1 (10000) 2024-04-23T01:56:11.417 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-23T01:56:11.475 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 7 16 1072 1056 1.17847 1.17188 0.124001 0.104993 2024-04-23T01:56:11.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:11.529+0000 7f55b379d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:11.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:11.541+0000 7f55b379d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:12.400 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.1 to 1 (10000) 2024-04-23T01:56:12.419 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-23T01:56:12.475 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 8 16 1216 1200 1.17178 1.125 0.0928103 0.105857 2024-04-23T01:56:12.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:12.538+0000 7faa6eb1d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:12.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:12.552+0000 7faa6eb1d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:12.870 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:56:13.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:13.004+0000 7f306bc6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:13.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:13.011+0000 7f306bc6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:13.400 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.2 to 1 (10000) 2024-04-23T01:56:13.422 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-23T01:56:13.475 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 9 16 1349 1333 1.15703 1.03906 0.120099 0.10684 2024-04-23T01:56:13.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:13.548+0000 7fc2f611d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:13.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:13.564+0000 7fc2f611d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:14.226 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.3 to 1 (10000) 2024-04-23T01:56:14.244 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-23T01:56:14.244 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:56:14.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:14.359+0000 7f682566d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:14.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:14.369+0000 7f682566d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:14.475 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 10 16 1467 1451 1.1335 0.921875 0.22794 0.109267 2024-04-23T01:56:14.695 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:56:14.696 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:56:14.715 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-04-23T01:56:14.225754+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":9495072,"num_objects":140,"num_object_clones":0,"num_object_copies":420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":1224,"num_write_kb":10590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1179,"ondisk_log_size":1179,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":418064,"kb_used_data":15088,"kb_used_omap":0,"kb_used_meta":402944,"kb_avail":377069296,"statfs":{"total":386547056640,"available":386118959104,"internally_reserved":0,"allocated":15450112,"data_stored":28652066,"data_compressed":1432938,"data_compressed_allocated":13160448,"data_compressed_original":27795456,"omap_allocated":0,"internal_metadata":412614656},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6184960,"num_objects":95,"num_object_clones":0,"num_object_copies":285,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":755,"num_write_kb":6040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.549412"},"pg_stats":[{"pgid":"9.7","version":"64'120","reported_seq":136,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.535400+0000","last_change":"2024-04-23T01:56:03.683930+0000","last_active":"2024-04-23T01:56:11.535400+0000","last_peered":"2024-04-23T01:56:11.535400+0000","last_clean":"2024-04-23T01:56:11.535400+0000","last_became_active":"2024-04-23T01:56:03.675554+0000","last_became_peered":"2024-04-23T01:56:03.675554+0000","last_unstale":"2024-04-23T01:56:11.535400+0000","last_undegraded":"2024-04-23T01:56:11.535400+0000","last_fullsized":"2024-04-23T01:56:11.535400+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":991232,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121,"num_write_kb":968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"64'96","reported_seq":111,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.575035+0000","last_change":"2024-04-23T01:56:03.552647+0000","last_active":"2024-04-23T01:56:11.575035+0000","last_peered":"2024-04-23T01:56:11.575035+0000","last_clean":"2024-04-23T01:56:11.575035+0000","last_became_active":"2024-04-23T01:56:03.541685+0000","last_became_peered":"2024-04-23T01:56:03.541685+0000","last_unstale":"2024-04-23T01:56:11.575035+0000","last_undegraded":"2024-04-23T01:56:11.575035+0000","last_fullsized":"2024-04-23T01:56:11.575035+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":786432,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96,"num_write_kb":768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"64'88","reported_seq":103,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.431339+0000","last_change":"2024-04-23T01:56:03.515568+0000","last_active":"2024-04-23T01:56:11.431339+0000","last_peered":"2024-04-23T01:56:11.431339+0000","last_clean":"2024-04-23T01:56:11.431339+0000","last_became_active":"2024-04-23T01:56:03.510354+0000","last_became_peered":"2024-04-23T01:56:03.510354+0000","last_unstale":"2024-04-23T01:56:11.431339+0000","last_undegraded":"2024-04-23T01:56:11.431339+0000","last_fullsized":"2024-04-23T01:56:11.431339+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":720896,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.4","version":"64'72","reported_seq":87,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.437106+0000","last_change":"2024-04-23T01:56:03.647359+0000","last_active":"2024-04-23T01:56:11.437106+0000","last_peered":"2024-04-23T01:56:11.437106+0000","last_clean":"2024-04-23T01:56:11.437106+0000","last_became_active":"2024-04-23T01:56:03.642592+0000","last_became_peered":"2024-04-23T01:56:03.642592+0000","last_unstale":"2024-04-23T01:56:11.437106+0000","last_undegraded":"2024-04-23T01:56:11.437106+0000","last_fullsized":"2024-04-23T01:56:11.437106+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"65'60","reported_seq":76,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.682692+0000","last_change":"2024-04-23T01:56:03.566150+0000","last_active":"2024-04-23T01:56:11.682692+0000","last_peered":"2024-04-23T01:56:11.682692+0000","last_clean":"2024-04-23T01:56:11.682692+0000","last_became_active":"2024-04-23T01:56:03.553086+0000","last_became_peered":"2024-04-23T01:56:03.553086+0000","last_unstale":"2024-04-23T01:56:11.682692+0000","last_undegraded":"2024-04-23T01:56:11.682692+0000","last_fullsized":"2024-04-23T01:56:11.682692+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":499712,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61,"num_write_kb":488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"64'56","reported_seq":71,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.581160+0000","last_change":"2024-04-23T01:56:03.605892+0000","last_active":"2024-04-23T01:56:11.581160+0000","last_peered":"2024-04-23T01:56:11.581160+0000","last_clean":"2024-04-23T01:56:11.581160+0000","last_became_active":"2024-04-23T01:56:03.601950+0000","last_became_peered":"2024-04-23T01:56:03.601950+0000","last_unstale":"2024-04-23T01:56:11.581160+0000","last_undegraded":"2024-04-23T01:56:11.581160+0000","last_fullsized":"2024-04-23T01:56:11.581160+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458752,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"64'48","reported_seq":63,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.529170+0000","last_change":"2024-04-23T01:56:03.618761+0000","last_active":"2024-04-23T01:56:11.529170+0000","last_peered":"2024-04-23T01:56:11.529170+0000","last_clean":"2024-04-23T01:56:11.529170+0000","last_became_active":"2024-04-23T01:56:03.602521+0000","last_became_peered":"2024-04-23T01:56:03.602521+0000","last_unstale":"2024-04-23T01:56:11.529170+0000","last_undegraded":"2024-04-23T01:56:11.529170+0000","last_fullsized":"2024-04-23T01:56:11.529170+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"65'79","reported_seq":95,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.744077+0000","last_change":"2024-04-23T01:56:03.544149+0000","last_active":"2024-04-23T01:56:11.744077+0000","last_peered":"2024-04-23T01:56:11.744077+0000","last_clean":"2024-04-23T01:56:11.744077+0000","last_became_active":"2024-04-23T01:56:03.539589+0000","last_became_peered":"2024-04-23T01:56:03.539589+0000","last_unstale":"2024-04-23T01:56:11.744077+0000","last_undegraded":"2024-04-23T01:56:11.744077+0000","last_fullsized":"2024-04-23T01:56:11.744077+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"64'80","reported_seq":95,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.544410+0000","last_change":"2024-04-23T01:56:03.635647+0000","last_active":"2024-04-23T01:56:11.544410+0000","last_peered":"2024-04-23T01:56:11.544410+0000","last_clean":"2024-04-23T01:56:11.544410+0000","last_became_active":"2024-04-23T01:56:03.627392+0000","last_became_peered":"2024-04-23T01:56:03.627392+0000","last_unstale":"2024-04-23T01:56:11.544410+0000","last_undegraded":"2024-04-23T01:56:11.544410+0000","last_fullsized":"2024-04-23T01:56:11.544410+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":282,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.551486+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:56:11.551486+0000","last_peered":"2024-04-23T01:56:11.551486+0000","last_clean":"2024-04-23T01:56:11.551486+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:56:11.551486+0000","last_undegraded":"2024-04-23T01:56:11.551486+0000","last_fullsized":"2024-04-23T01:56:11.551486+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.584234+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:56:11.584234+0000","last_peered":"2024-04-23T01:56:11.584234+0000","last_clean":"2024-04-23T01:56:11.584234+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:56:11.584234+0000","last_undegraded":"2024-04-23T01:56:11.584234+0000","last_fullsized":"2024-04-23T01:56:11.584234+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.b","version":"64'32","reported_seq":47,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.545397+0000","last_change":"2024-04-23T01:56:03.687235+0000","last_active":"2024-04-23T01:56:11.545397+0000","last_peered":"2024-04-23T01:56:11.545397+0000","last_clean":"2024-04-23T01:56:11.545397+0000","last_became_active":"2024-04-23T01:56:03.683119+0000","last_became_peered":"2024-04-23T01:56:03.683119+0000","last_unstale":"2024-04-23T01:56:11.545397+0000","last_undegraded":"2024-04-23T01:56:11.545397+0000","last_fullsized":"2024-04-23T01:56:11.545397+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":143,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.534076+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:56:11.534076+0000","last_peered":"2024-04-23T01:56:11.534076+0000","last_clean":"2024-04-23T01:56:11.534076+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:56:11.534076+0000","last_undegraded":"2024-04-23T01:56:11.534076+0000","last_fullsized":"2024-04-23T01:56:11.534076+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"64'80","reported_seq":95,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.424102+0000","last_change":"2024-04-23T01:56:03.560440+0000","last_active":"2024-04-23T01:56:11.424102+0000","last_peered":"2024-04-23T01:56:11.424102+0000","last_clean":"2024-04-23T01:56:11.424102+0000","last_became_active":"2024-04-23T01:56:03.552840+0000","last_became_peered":"2024-04-23T01:56:03.552840+0000","last_unstale":"2024-04-23T01:56:11.424102+0000","last_undegraded":"2024-04-23T01:56:11.424102+0000","last_fullsized":"2024-04-23T01:56:11.424102+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.577893+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:56:11.577893+0000","last_peered":"2024-04-23T01:56:11.577893+0000","last_clean":"2024-04-23T01:56:11.577893+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:56:11.577893+0000","last_undegraded":"2024-04-23T01:56:11.577893+0000","last_fullsized":"2024-04-23T01:56:11.577893+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"64'40","reported_seq":56,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.549547+0000","last_change":"2024-04-23T01:56:03.597772+0000","last_active":"2024-04-23T01:56:11.549547+0000","last_peered":"2024-04-23T01:56:11.549547+0000","last_clean":"2024-04-23T01:56:11.549547+0000","last_became_active":"2024-04-23T01:56:03.589730+0000","last_became_peered":"2024-04-23T01:56:03.589730+0000","last_unstale":"2024-04-23T01:56:11.549547+0000","last_undegraded":"2024-04-23T01:56:11.549547+0000","last_fullsized":"2024-04-23T01:56:11.549547+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335872,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":143,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.550539+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:56:11.550539+0000","last_peered":"2024-04-23T01:56:11.550539+0000","last_clean":"2024-04-23T01:56:11.550539+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:56:11.550539+0000","last_undegraded":"2024-04-23T01:56:11.550539+0000","last_fullsized":"2024-04-23T01:56:11.550539+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.f","version":"65'55","reported_seq":71,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.637771+0000","last_change":"2024-04-23T01:56:03.654933+0000","last_active":"2024-04-23T01:56:11.637771+0000","last_peered":"2024-04-23T01:56:11.637771+0000","last_clean":"2024-04-23T01:56:11.637771+0000","last_became_active":"2024-04-23T01:56:03.641185+0000","last_became_peered":"2024-04-23T01:56:03.641185+0000","last_unstale":"2024-04-23T01:56:11.637771+0000","last_undegraded":"2024-04-23T01:56:11.637771+0000","last_fullsized":"2024-04-23T01:56:11.637771+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458752,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":149,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.440825+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:56:11.440825+0000","last_peered":"2024-04-23T01:56:11.440825+0000","last_clean":"2024-04-23T01:56:11.440825+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:56:11.440825+0000","last_undegraded":"2024-04-23T01:56:11.440825+0000","last_fullsized":"2024-04-23T01:56:11.440825+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.e","version":"64'64","reported_seq":79,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.528063+0000","last_change":"2024-04-23T01:56:03.625197+0000","last_active":"2024-04-23T01:56:11.528063+0000","last_peered":"2024-04-23T01:56:11.528063+0000","last_clean":"2024-04-23T01:56:11.528063+0000","last_became_active":"2024-04-23T01:56:03.611498+0000","last_became_peered":"2024-04-23T01:56:03.611498+0000","last_unstale":"2024-04-23T01:56:11.528063+0000","last_undegraded":"2024-04-23T01:56:11.528063+0000","last_fullsized":"2024-04-23T01:56:11.528063+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":149,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.549834+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:56:11.549834+0000","last_peered":"2024-04-23T01:56:11.549834+0000","last_clean":"2024-04-23T01:56:11.549834+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:56:11.549834+0000","last_undegraded":"2024-04-23T01:56:11.549834+0000","last_fullsized":"2024-04-23T01:56:11.549834+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"64'48","reported_seq":63,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.539613+0000","last_change":"2024-04-23T01:56:03.612020+0000","last_active":"2024-04-23T01:56:11.539613+0000","last_peered":"2024-04-23T01:56:11.539613+0000","last_clean":"2024-04-23T01:56:11.539613+0000","last_became_active":"2024-04-23T01:56:03.605401+0000","last_became_peered":"2024-04-23T01:56:03.605401+0000","last_unstale":"2024-04-23T01:56:11.539613+0000","last_undegraded":"2024-04-23T01:56:11.539613+0000","last_fullsized":"2024-04-23T01:56:11.539613+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":149,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.442901+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:56:11.442901+0000","last_peered":"2024-04-23T01:56:11.442901+0000","last_clean":"2024-04-23T01:56:11.442901+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:56:11.442901+0000","last_undegraded":"2024-04-23T01:56:11.442901+0000","last_fullsized":"2024-04-23T01:56:11.442901+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.c","version":"64'64","reported_seq":79,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.564141+0000","last_change":"2024-04-23T01:56:03.574136+0000","last_active":"2024-04-23T01:56:11.564141+0000","last_peered":"2024-04-23T01:56:11.564141+0000","last_clean":"2024-04-23T01:56:11.564141+0000","last_became_active":"2024-04-23T01:56:03.566135+0000","last_became_peered":"2024-04-23T01:56:03.566135+0000","last_unstale":"2024-04-23T01:56:11.564141+0000","last_undegraded":"2024-04-23T01:56:11.564141+0000","last_fullsized":"2024-04-23T01:56:11.564141+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.569818+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:56:11.569818+0000","last_peered":"2024-04-23T01:56:11.569818+0000","last_clean":"2024-04-23T01:56:11.569818+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:56:11.569818+0000","last_undegraded":"2024-04-23T01:56:11.569818+0000","last_fullsized":"2024-04-23T01:56:11.569818+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":8904704,"num_objects":138,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1087,"num_write_kb":8696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":13320192,"data_stored":26640384,"data_compressed":1372266,"data_compressed_allocated":13320192,"data_compressed_original":26640384,"omap_allocated":0,"internal_metadata":0},"log_size":1082,"ondisk_log_size":1082,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706328,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":103080,"kb_used_data":4064,"kb_used_omap":0,"kb_used_meta":99008,"kb_avail":94268760,"statfs":{"total":96636764160,"available":96531210240,"internally_reserved":0,"allocated":4161536,"data_stored":7884474,"data_compressed":395353,"data_compressed_allocated":3588096,"data_compressed_original":7667712,"omap_allocated":0,"internal_metadata":101384192},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706328,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":92512,"kb_used_data":3224,"kb_used_omap":0,"kb_used_meta":89280,"kb_avail":94279328,"statfs":{"total":96636764160,"available":96542031872,"internally_reserved":0,"allocated":3301376,"data_stored":6164154,"data_compressed":306741,"data_compressed_allocated":2727936,"data_compressed_original":5947392,"omap_allocated":0,"internal_metadata":91422720},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706328,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":114776,"kb_used_data":3280,"kb_used_omap":0,"kb_used_meta":111488,"kb_avail":94257064,"statfs":{"total":96636764160,"available":96519233536,"internally_reserved":0,"allocated":3358720,"data_stored":5798215,"data_compressed":288211,"data_compressed_allocated":2797568,"data_compressed_original":5595136,"omap_allocated":0,"internal_metadata":114163712},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706328,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":107696,"kb_used_data":4520,"kb_used_omap":0,"kb_used_meta":103168,"kb_avail":94264144,"statfs":{"total":96636764160,"available":96526483456,"internally_reserved":0,"allocated":4628480,"data_stored":8805223,"data_compressed":442633,"data_compressed_allocated":4046848,"data_compressed_original":8585216,"omap_allocated":0,"internal_metadata":105644032},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4022272,"data_stored":8044544,"data_compressed":414372,"data_compressed_allocated":4022272,"data_compressed_original":8044544,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2838528,"data_stored":5677056,"data_compressed":292441,"data_compressed_allocated":2838528,"data_compressed_original":5677056,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2813952,"data_stored":5627904,"data_compressed":289901,"data_compressed_allocated":2813952,"data_compressed_original":5627904,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3645440,"data_stored":7290880,"data_compressed":375552,"data_compressed_allocated":3645440,"data_compressed_original":7290880,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:56:14.717 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:56:14.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:14.828+0000 7f345486d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:14.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:14.838+0000 7f345486d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:15.076 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:56:15.077 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:56:15.091 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-04-23T01:56:14.225754+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":9495072,"num_objects":140,"num_object_clones":0,"num_object_copies":420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":1224,"num_write_kb":10590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1179,"ondisk_log_size":1179,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":418064,"kb_used_data":15088,"kb_used_omap":0,"kb_used_meta":402944,"kb_avail":377069296,"statfs":{"total":386547056640,"available":386118959104,"internally_reserved":0,"allocated":15450112,"data_stored":28652066,"data_compressed":1432938,"data_compressed_allocated":13160448,"data_compressed_original":27795456,"omap_allocated":0,"internal_metadata":412614656},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6184960,"num_objects":95,"num_object_clones":0,"num_object_copies":285,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":755,"num_write_kb":6040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.549412"},"pg_stats":[{"pgid":"9.7","version":"64'120","reported_seq":136,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.535400+0000","last_change":"2024-04-23T01:56:03.683930+0000","last_active":"2024-04-23T01:56:11.535400+0000","last_peered":"2024-04-23T01:56:11.535400+0000","last_clean":"2024-04-23T01:56:11.535400+0000","last_became_active":"2024-04-23T01:56:03.675554+0000","last_became_peered":"2024-04-23T01:56:03.675554+0000","last_unstale":"2024-04-23T01:56:11.535400+0000","last_undegraded":"2024-04-23T01:56:11.535400+0000","last_fullsized":"2024-04-23T01:56:11.535400+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":991232,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121,"num_write_kb":968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"64'96","reported_seq":111,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.575035+0000","last_change":"2024-04-23T01:56:03.552647+0000","last_active":"2024-04-23T01:56:11.575035+0000","last_peered":"2024-04-23T01:56:11.575035+0000","last_clean":"2024-04-23T01:56:11.575035+0000","last_became_active":"2024-04-23T01:56:03.541685+0000","last_became_peered":"2024-04-23T01:56:03.541685+0000","last_unstale":"2024-04-23T01:56:11.575035+0000","last_undegraded":"2024-04-23T01:56:11.575035+0000","last_fullsized":"2024-04-23T01:56:11.575035+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":786432,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96,"num_write_kb":768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"64'88","reported_seq":103,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.431339+0000","last_change":"2024-04-23T01:56:03.515568+0000","last_active":"2024-04-23T01:56:11.431339+0000","last_peered":"2024-04-23T01:56:11.431339+0000","last_clean":"2024-04-23T01:56:11.431339+0000","last_became_active":"2024-04-23T01:56:03.510354+0000","last_became_peered":"2024-04-23T01:56:03.510354+0000","last_unstale":"2024-04-23T01:56:11.431339+0000","last_undegraded":"2024-04-23T01:56:11.431339+0000","last_fullsized":"2024-04-23T01:56:11.431339+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":720896,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.4","version":"64'72","reported_seq":87,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.437106+0000","last_change":"2024-04-23T01:56:03.647359+0000","last_active":"2024-04-23T01:56:11.437106+0000","last_peered":"2024-04-23T01:56:11.437106+0000","last_clean":"2024-04-23T01:56:11.437106+0000","last_became_active":"2024-04-23T01:56:03.642592+0000","last_became_peered":"2024-04-23T01:56:03.642592+0000","last_unstale":"2024-04-23T01:56:11.437106+0000","last_undegraded":"2024-04-23T01:56:11.437106+0000","last_fullsized":"2024-04-23T01:56:11.437106+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"65'60","reported_seq":76,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.682692+0000","last_change":"2024-04-23T01:56:03.566150+0000","last_active":"2024-04-23T01:56:11.682692+0000","last_peered":"2024-04-23T01:56:11.682692+0000","last_clean":"2024-04-23T01:56:11.682692+0000","last_became_active":"2024-04-23T01:56:03.553086+0000","last_became_peered":"2024-04-23T01:56:03.553086+0000","last_unstale":"2024-04-23T01:56:11.682692+0000","last_undegraded":"2024-04-23T01:56:11.682692+0000","last_fullsized":"2024-04-23T01:56:11.682692+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":499712,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61,"num_write_kb":488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"64'56","reported_seq":71,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.581160+0000","last_change":"2024-04-23T01:56:03.605892+0000","last_active":"2024-04-23T01:56:11.581160+0000","last_peered":"2024-04-23T01:56:11.581160+0000","last_clean":"2024-04-23T01:56:11.581160+0000","last_became_active":"2024-04-23T01:56:03.601950+0000","last_became_peered":"2024-04-23T01:56:03.601950+0000","last_unstale":"2024-04-23T01:56:11.581160+0000","last_undegraded":"2024-04-23T01:56:11.581160+0000","last_fullsized":"2024-04-23T01:56:11.581160+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458752,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"64'48","reported_seq":63,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.529170+0000","last_change":"2024-04-23T01:56:03.618761+0000","last_active":"2024-04-23T01:56:11.529170+0000","last_peered":"2024-04-23T01:56:11.529170+0000","last_clean":"2024-04-23T01:56:11.529170+0000","last_became_active":"2024-04-23T01:56:03.602521+0000","last_became_peered":"2024-04-23T01:56:03.602521+0000","last_unstale":"2024-04-23T01:56:11.529170+0000","last_undegraded":"2024-04-23T01:56:11.529170+0000","last_fullsized":"2024-04-23T01:56:11.529170+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"65'79","reported_seq":95,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.744077+0000","last_change":"2024-04-23T01:56:03.544149+0000","last_active":"2024-04-23T01:56:11.744077+0000","last_peered":"2024-04-23T01:56:11.744077+0000","last_clean":"2024-04-23T01:56:11.744077+0000","last_became_active":"2024-04-23T01:56:03.539589+0000","last_became_peered":"2024-04-23T01:56:03.539589+0000","last_unstale":"2024-04-23T01:56:11.744077+0000","last_undegraded":"2024-04-23T01:56:11.744077+0000","last_fullsized":"2024-04-23T01:56:11.744077+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"64'80","reported_seq":95,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.544410+0000","last_change":"2024-04-23T01:56:03.635647+0000","last_active":"2024-04-23T01:56:11.544410+0000","last_peered":"2024-04-23T01:56:11.544410+0000","last_clean":"2024-04-23T01:56:11.544410+0000","last_became_active":"2024-04-23T01:56:03.627392+0000","last_became_peered":"2024-04-23T01:56:03.627392+0000","last_unstale":"2024-04-23T01:56:11.544410+0000","last_undegraded":"2024-04-23T01:56:11.544410+0000","last_fullsized":"2024-04-23T01:56:11.544410+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":282,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.551486+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:56:11.551486+0000","last_peered":"2024-04-23T01:56:11.551486+0000","last_clean":"2024-04-23T01:56:11.551486+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:56:11.551486+0000","last_undegraded":"2024-04-23T01:56:11.551486+0000","last_fullsized":"2024-04-23T01:56:11.551486+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.584234+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:56:11.584234+0000","last_peered":"2024-04-23T01:56:11.584234+0000","last_clean":"2024-04-23T01:56:11.584234+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:56:11.584234+0000","last_undegraded":"2024-04-23T01:56:11.584234+0000","last_fullsized":"2024-04-23T01:56:11.584234+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.b","version":"64'32","reported_seq":47,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.545397+0000","last_change":"2024-04-23T01:56:03.687235+0000","last_active":"2024-04-23T01:56:11.545397+0000","last_peered":"2024-04-23T01:56:11.545397+0000","last_clean":"2024-04-23T01:56:11.545397+0000","last_became_active":"2024-04-23T01:56:03.683119+0000","last_became_peered":"2024-04-23T01:56:03.683119+0000","last_unstale":"2024-04-23T01:56:11.545397+0000","last_undegraded":"2024-04-23T01:56:11.545397+0000","last_fullsized":"2024-04-23T01:56:11.545397+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":143,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.534076+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:56:11.534076+0000","last_peered":"2024-04-23T01:56:11.534076+0000","last_clean":"2024-04-23T01:56:11.534076+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:56:11.534076+0000","last_undegraded":"2024-04-23T01:56:11.534076+0000","last_fullsized":"2024-04-23T01:56:11.534076+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"64'80","reported_seq":95,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.424102+0000","last_change":"2024-04-23T01:56:03.560440+0000","last_active":"2024-04-23T01:56:11.424102+0000","last_peered":"2024-04-23T01:56:11.424102+0000","last_clean":"2024-04-23T01:56:11.424102+0000","last_became_active":"2024-04-23T01:56:03.552840+0000","last_became_peered":"2024-04-23T01:56:03.552840+0000","last_unstale":"2024-04-23T01:56:11.424102+0000","last_undegraded":"2024-04-23T01:56:11.424102+0000","last_fullsized":"2024-04-23T01:56:11.424102+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.577893+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:56:11.577893+0000","last_peered":"2024-04-23T01:56:11.577893+0000","last_clean":"2024-04-23T01:56:11.577893+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:56:11.577893+0000","last_undegraded":"2024-04-23T01:56:11.577893+0000","last_fullsized":"2024-04-23T01:56:11.577893+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"64'40","reported_seq":56,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.549547+0000","last_change":"2024-04-23T01:56:03.597772+0000","last_active":"2024-04-23T01:56:11.549547+0000","last_peered":"2024-04-23T01:56:11.549547+0000","last_clean":"2024-04-23T01:56:11.549547+0000","last_became_active":"2024-04-23T01:56:03.589730+0000","last_became_peered":"2024-04-23T01:56:03.589730+0000","last_unstale":"2024-04-23T01:56:11.549547+0000","last_undegraded":"2024-04-23T01:56:11.549547+0000","last_fullsized":"2024-04-23T01:56:11.549547+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335872,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":143,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.550539+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:56:11.550539+0000","last_peered":"2024-04-23T01:56:11.550539+0000","last_clean":"2024-04-23T01:56:11.550539+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:56:11.550539+0000","last_undegraded":"2024-04-23T01:56:11.550539+0000","last_fullsized":"2024-04-23T01:56:11.550539+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.f","version":"65'55","reported_seq":71,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.637771+0000","last_change":"2024-04-23T01:56:03.654933+0000","last_active":"2024-04-23T01:56:11.637771+0000","last_peered":"2024-04-23T01:56:11.637771+0000","last_clean":"2024-04-23T01:56:11.637771+0000","last_became_active":"2024-04-23T01:56:03.641185+0000","last_became_peered":"2024-04-23T01:56:03.641185+0000","last_unstale":"2024-04-23T01:56:11.637771+0000","last_undegraded":"2024-04-23T01:56:11.637771+0000","last_fullsized":"2024-04-23T01:56:11.637771+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458752,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":149,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.440825+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:56:11.440825+0000","last_peered":"2024-04-23T01:56:11.440825+0000","last_clean":"2024-04-23T01:56:11.440825+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:56:11.440825+0000","last_undegraded":"2024-04-23T01:56:11.440825+0000","last_fullsized":"2024-04-23T01:56:11.440825+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.e","version":"64'64","reported_seq":79,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.528063+0000","last_change":"2024-04-23T01:56:03.625197+0000","last_active":"2024-04-23T01:56:11.528063+0000","last_peered":"2024-04-23T01:56:11.528063+0000","last_clean":"2024-04-23T01:56:11.528063+0000","last_became_active":"2024-04-23T01:56:03.611498+0000","last_became_peered":"2024-04-23T01:56:03.611498+0000","last_unstale":"2024-04-23T01:56:11.528063+0000","last_undegraded":"2024-04-23T01:56:11.528063+0000","last_fullsized":"2024-04-23T01:56:11.528063+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":149,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.549834+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:56:11.549834+0000","last_peered":"2024-04-23T01:56:11.549834+0000","last_clean":"2024-04-23T01:56:11.549834+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:56:11.549834+0000","last_undegraded":"2024-04-23T01:56:11.549834+0000","last_fullsized":"2024-04-23T01:56:11.549834+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"64'48","reported_seq":63,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.539613+0000","last_change":"2024-04-23T01:56:03.612020+0000","last_active":"2024-04-23T01:56:11.539613+0000","last_peered":"2024-04-23T01:56:11.539613+0000","last_clean":"2024-04-23T01:56:11.539613+0000","last_became_active":"2024-04-23T01:56:03.605401+0000","last_became_peered":"2024-04-23T01:56:03.605401+0000","last_unstale":"2024-04-23T01:56:11.539613+0000","last_undegraded":"2024-04-23T01:56:11.539613+0000","last_fullsized":"2024-04-23T01:56:11.539613+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":149,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.442901+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:56:11.442901+0000","last_peered":"2024-04-23T01:56:11.442901+0000","last_clean":"2024-04-23T01:56:11.442901+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:56:11.442901+0000","last_undegraded":"2024-04-23T01:56:11.442901+0000","last_fullsized":"2024-04-23T01:56:11.442901+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.c","version":"64'64","reported_seq":79,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.564141+0000","last_change":"2024-04-23T01:56:03.574136+0000","last_active":"2024-04-23T01:56:11.564141+0000","last_peered":"2024-04-23T01:56:11.564141+0000","last_clean":"2024-04-23T01:56:11.564141+0000","last_became_active":"2024-04-23T01:56:03.566135+0000","last_became_peered":"2024-04-23T01:56:03.566135+0000","last_unstale":"2024-04-23T01:56:11.564141+0000","last_undegraded":"2024-04-23T01:56:11.564141+0000","last_fullsized":"2024-04-23T01:56:11.564141+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.569818+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:56:11.569818+0000","last_peered":"2024-04-23T01:56:11.569818+0000","last_clean":"2024-04-23T01:56:11.569818+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:56:11.569818+0000","last_undegraded":"2024-04-23T01:56:11.569818+0000","last_fullsized":"2024-04-23T01:56:11.569818+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":8904704,"num_objects":138,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1087,"num_write_kb":8696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":13320192,"data_stored":26640384,"data_compressed":1372266,"data_compressed_allocated":13320192,"data_compressed_original":26640384,"omap_allocated":0,"internal_metadata":0},"log_size":1082,"ondisk_log_size":1082,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706328,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":103080,"kb_used_data":4064,"kb_used_omap":0,"kb_used_meta":99008,"kb_avail":94268760,"statfs":{"total":96636764160,"available":96531210240,"internally_reserved":0,"allocated":4161536,"data_stored":7884474,"data_compressed":395353,"data_compressed_allocated":3588096,"data_compressed_original":7667712,"omap_allocated":0,"internal_metadata":101384192},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706328,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":92512,"kb_used_data":3224,"kb_used_omap":0,"kb_used_meta":89280,"kb_avail":94279328,"statfs":{"total":96636764160,"available":96542031872,"internally_reserved":0,"allocated":3301376,"data_stored":6164154,"data_compressed":306741,"data_compressed_allocated":2727936,"data_compressed_original":5947392,"omap_allocated":0,"internal_metadata":91422720},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706328,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":114776,"kb_used_data":3280,"kb_used_omap":0,"kb_used_meta":111488,"kb_avail":94257064,"statfs":{"total":96636764160,"available":96519233536,"internally_reserved":0,"allocated":3358720,"data_stored":5798215,"data_compressed":288211,"data_compressed_allocated":2797568,"data_compressed_original":5595136,"omap_allocated":0,"internal_metadata":114163712},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706328,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":107696,"kb_used_data":4520,"kb_used_omap":0,"kb_used_meta":103168,"kb_avail":94264144,"statfs":{"total":96636764160,"available":96526483456,"internally_reserved":0,"allocated":4628480,"data_stored":8805223,"data_compressed":442633,"data_compressed_allocated":4046848,"data_compressed_original":8585216,"omap_allocated":0,"internal_metadata":105644032},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4022272,"data_stored":8044544,"data_compressed":414372,"data_compressed_allocated":4022272,"data_compressed_original":8044544,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2838528,"data_stored":5677056,"data_compressed":292441,"data_compressed_allocated":2838528,"data_compressed_original":5677056,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2813952,"data_stored":5627904,"data_compressed":289901,"data_compressed_allocated":2813952,"data_compressed_original":5627904,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3645440,"data_stored":7290880,"data_compressed":375552,"data_compressed_allocated":3645440,"data_compressed_original":7290880,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:56:15.093 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-23T01:56:15.093 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:56:15.093 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:56:15.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:15.234+0000 7f305179d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:15.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:15.244+0000 7f305179d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:15.261 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:56:15.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:15.395+0000 7f5ef195d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:15.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:15.404+0000 7f5ef195d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:15.475 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 11 16 1586 1570 1.11497 0.929688 0.236339 0.111246 2024-04-23T01:56:15.484 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:56:15.484 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:56:15.503 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-04-23T01:56:14.225754+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":9495072,"num_objects":140,"num_object_clones":0,"num_object_copies":420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":1224,"num_write_kb":10590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1179,"ondisk_log_size":1179,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":418064,"kb_used_data":15088,"kb_used_omap":0,"kb_used_meta":402944,"kb_avail":377069296,"statfs":{"total":386547056640,"available":386118959104,"internally_reserved":0,"allocated":15450112,"data_stored":28652066,"data_compressed":1432938,"data_compressed_allocated":13160448,"data_compressed_original":27795456,"omap_allocated":0,"internal_metadata":412614656},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6184960,"num_objects":95,"num_object_clones":0,"num_object_copies":285,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":755,"num_write_kb":6040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.549412"},"pg_stats":[{"pgid":"9.7","version":"64'120","reported_seq":136,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.535400+0000","last_change":"2024-04-23T01:56:03.683930+0000","last_active":"2024-04-23T01:56:11.535400+0000","last_peered":"2024-04-23T01:56:11.535400+0000","last_clean":"2024-04-23T01:56:11.535400+0000","last_became_active":"2024-04-23T01:56:03.675554+0000","last_became_peered":"2024-04-23T01:56:03.675554+0000","last_unstale":"2024-04-23T01:56:11.535400+0000","last_undegraded":"2024-04-23T01:56:11.535400+0000","last_fullsized":"2024-04-23T01:56:11.535400+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":991232,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121,"num_write_kb":968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"64'96","reported_seq":111,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.575035+0000","last_change":"2024-04-23T01:56:03.552647+0000","last_active":"2024-04-23T01:56:11.575035+0000","last_peered":"2024-04-23T01:56:11.575035+0000","last_clean":"2024-04-23T01:56:11.575035+0000","last_became_active":"2024-04-23T01:56:03.541685+0000","last_became_peered":"2024-04-23T01:56:03.541685+0000","last_unstale":"2024-04-23T01:56:11.575035+0000","last_undegraded":"2024-04-23T01:56:11.575035+0000","last_fullsized":"2024-04-23T01:56:11.575035+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":786432,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96,"num_write_kb":768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"64'88","reported_seq":103,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.431339+0000","last_change":"2024-04-23T01:56:03.515568+0000","last_active":"2024-04-23T01:56:11.431339+0000","last_peered":"2024-04-23T01:56:11.431339+0000","last_clean":"2024-04-23T01:56:11.431339+0000","last_became_active":"2024-04-23T01:56:03.510354+0000","last_became_peered":"2024-04-23T01:56:03.510354+0000","last_unstale":"2024-04-23T01:56:11.431339+0000","last_undegraded":"2024-04-23T01:56:11.431339+0000","last_fullsized":"2024-04-23T01:56:11.431339+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":720896,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.4","version":"64'72","reported_seq":87,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.437106+0000","last_change":"2024-04-23T01:56:03.647359+0000","last_active":"2024-04-23T01:56:11.437106+0000","last_peered":"2024-04-23T01:56:11.437106+0000","last_clean":"2024-04-23T01:56:11.437106+0000","last_became_active":"2024-04-23T01:56:03.642592+0000","last_became_peered":"2024-04-23T01:56:03.642592+0000","last_unstale":"2024-04-23T01:56:11.437106+0000","last_undegraded":"2024-04-23T01:56:11.437106+0000","last_fullsized":"2024-04-23T01:56:11.437106+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"65'60","reported_seq":76,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.682692+0000","last_change":"2024-04-23T01:56:03.566150+0000","last_active":"2024-04-23T01:56:11.682692+0000","last_peered":"2024-04-23T01:56:11.682692+0000","last_clean":"2024-04-23T01:56:11.682692+0000","last_became_active":"2024-04-23T01:56:03.553086+0000","last_became_peered":"2024-04-23T01:56:03.553086+0000","last_unstale":"2024-04-23T01:56:11.682692+0000","last_undegraded":"2024-04-23T01:56:11.682692+0000","last_fullsized":"2024-04-23T01:56:11.682692+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":499712,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61,"num_write_kb":488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"64'56","reported_seq":71,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.581160+0000","last_change":"2024-04-23T01:56:03.605892+0000","last_active":"2024-04-23T01:56:11.581160+0000","last_peered":"2024-04-23T01:56:11.581160+0000","last_clean":"2024-04-23T01:56:11.581160+0000","last_became_active":"2024-04-23T01:56:03.601950+0000","last_became_peered":"2024-04-23T01:56:03.601950+0000","last_unstale":"2024-04-23T01:56:11.581160+0000","last_undegraded":"2024-04-23T01:56:11.581160+0000","last_fullsized":"2024-04-23T01:56:11.581160+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458752,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"64'48","reported_seq":63,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.529170+0000","last_change":"2024-04-23T01:56:03.618761+0000","last_active":"2024-04-23T01:56:11.529170+0000","last_peered":"2024-04-23T01:56:11.529170+0000","last_clean":"2024-04-23T01:56:11.529170+0000","last_became_active":"2024-04-23T01:56:03.602521+0000","last_became_peered":"2024-04-23T01:56:03.602521+0000","last_unstale":"2024-04-23T01:56:11.529170+0000","last_undegraded":"2024-04-23T01:56:11.529170+0000","last_fullsized":"2024-04-23T01:56:11.529170+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"65'79","reported_seq":95,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.744077+0000","last_change":"2024-04-23T01:56:03.544149+0000","last_active":"2024-04-23T01:56:11.744077+0000","last_peered":"2024-04-23T01:56:11.744077+0000","last_clean":"2024-04-23T01:56:11.744077+0000","last_became_active":"2024-04-23T01:56:03.539589+0000","last_became_peered":"2024-04-23T01:56:03.539589+0000","last_unstale":"2024-04-23T01:56:11.744077+0000","last_undegraded":"2024-04-23T01:56:11.744077+0000","last_fullsized":"2024-04-23T01:56:11.744077+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"64'80","reported_seq":95,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.544410+0000","last_change":"2024-04-23T01:56:03.635647+0000","last_active":"2024-04-23T01:56:11.544410+0000","last_peered":"2024-04-23T01:56:11.544410+0000","last_clean":"2024-04-23T01:56:11.544410+0000","last_became_active":"2024-04-23T01:56:03.627392+0000","last_became_peered":"2024-04-23T01:56:03.627392+0000","last_unstale":"2024-04-23T01:56:11.544410+0000","last_undegraded":"2024-04-23T01:56:11.544410+0000","last_fullsized":"2024-04-23T01:56:11.544410+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":282,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.551486+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:56:11.551486+0000","last_peered":"2024-04-23T01:56:11.551486+0000","last_clean":"2024-04-23T01:56:11.551486+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:56:11.551486+0000","last_undegraded":"2024-04-23T01:56:11.551486+0000","last_fullsized":"2024-04-23T01:56:11.551486+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.584234+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:56:11.584234+0000","last_peered":"2024-04-23T01:56:11.584234+0000","last_clean":"2024-04-23T01:56:11.584234+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:56:11.584234+0000","last_undegraded":"2024-04-23T01:56:11.584234+0000","last_fullsized":"2024-04-23T01:56:11.584234+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.b","version":"64'32","reported_seq":47,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.545397+0000","last_change":"2024-04-23T01:56:03.687235+0000","last_active":"2024-04-23T01:56:11.545397+0000","last_peered":"2024-04-23T01:56:11.545397+0000","last_clean":"2024-04-23T01:56:11.545397+0000","last_became_active":"2024-04-23T01:56:03.683119+0000","last_became_peered":"2024-04-23T01:56:03.683119+0000","last_unstale":"2024-04-23T01:56:11.545397+0000","last_undegraded":"2024-04-23T01:56:11.545397+0000","last_fullsized":"2024-04-23T01:56:11.545397+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":143,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.534076+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:56:11.534076+0000","last_peered":"2024-04-23T01:56:11.534076+0000","last_clean":"2024-04-23T01:56:11.534076+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:56:11.534076+0000","last_undegraded":"2024-04-23T01:56:11.534076+0000","last_fullsized":"2024-04-23T01:56:11.534076+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"64'80","reported_seq":95,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.424102+0000","last_change":"2024-04-23T01:56:03.560440+0000","last_active":"2024-04-23T01:56:11.424102+0000","last_peered":"2024-04-23T01:56:11.424102+0000","last_clean":"2024-04-23T01:56:11.424102+0000","last_became_active":"2024-04-23T01:56:03.552840+0000","last_became_peered":"2024-04-23T01:56:03.552840+0000","last_unstale":"2024-04-23T01:56:11.424102+0000","last_undegraded":"2024-04-23T01:56:11.424102+0000","last_fullsized":"2024-04-23T01:56:11.424102+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.577893+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:56:11.577893+0000","last_peered":"2024-04-23T01:56:11.577893+0000","last_clean":"2024-04-23T01:56:11.577893+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:56:11.577893+0000","last_undegraded":"2024-04-23T01:56:11.577893+0000","last_fullsized":"2024-04-23T01:56:11.577893+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"64'40","reported_seq":56,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.549547+0000","last_change":"2024-04-23T01:56:03.597772+0000","last_active":"2024-04-23T01:56:11.549547+0000","last_peered":"2024-04-23T01:56:11.549547+0000","last_clean":"2024-04-23T01:56:11.549547+0000","last_became_active":"2024-04-23T01:56:03.589730+0000","last_became_peered":"2024-04-23T01:56:03.589730+0000","last_unstale":"2024-04-23T01:56:11.549547+0000","last_undegraded":"2024-04-23T01:56:11.549547+0000","last_fullsized":"2024-04-23T01:56:11.549547+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335872,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":143,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.550539+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:56:11.550539+0000","last_peered":"2024-04-23T01:56:11.550539+0000","last_clean":"2024-04-23T01:56:11.550539+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:56:11.550539+0000","last_undegraded":"2024-04-23T01:56:11.550539+0000","last_fullsized":"2024-04-23T01:56:11.550539+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.f","version":"65'55","reported_seq":71,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.637771+0000","last_change":"2024-04-23T01:56:03.654933+0000","last_active":"2024-04-23T01:56:11.637771+0000","last_peered":"2024-04-23T01:56:11.637771+0000","last_clean":"2024-04-23T01:56:11.637771+0000","last_became_active":"2024-04-23T01:56:03.641185+0000","last_became_peered":"2024-04-23T01:56:03.641185+0000","last_unstale":"2024-04-23T01:56:11.637771+0000","last_undegraded":"2024-04-23T01:56:11.637771+0000","last_fullsized":"2024-04-23T01:56:11.637771+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458752,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":149,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.440825+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:56:11.440825+0000","last_peered":"2024-04-23T01:56:11.440825+0000","last_clean":"2024-04-23T01:56:11.440825+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:56:11.440825+0000","last_undegraded":"2024-04-23T01:56:11.440825+0000","last_fullsized":"2024-04-23T01:56:11.440825+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.e","version":"64'64","reported_seq":79,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.528063+0000","last_change":"2024-04-23T01:56:03.625197+0000","last_active":"2024-04-23T01:56:11.528063+0000","last_peered":"2024-04-23T01:56:11.528063+0000","last_clean":"2024-04-23T01:56:11.528063+0000","last_became_active":"2024-04-23T01:56:03.611498+0000","last_became_peered":"2024-04-23T01:56:03.611498+0000","last_unstale":"2024-04-23T01:56:11.528063+0000","last_undegraded":"2024-04-23T01:56:11.528063+0000","last_fullsized":"2024-04-23T01:56:11.528063+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":149,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.549834+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:56:11.549834+0000","last_peered":"2024-04-23T01:56:11.549834+0000","last_clean":"2024-04-23T01:56:11.549834+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:56:11.549834+0000","last_undegraded":"2024-04-23T01:56:11.549834+0000","last_fullsized":"2024-04-23T01:56:11.549834+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"64'48","reported_seq":63,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.539613+0000","last_change":"2024-04-23T01:56:03.612020+0000","last_active":"2024-04-23T01:56:11.539613+0000","last_peered":"2024-04-23T01:56:11.539613+0000","last_clean":"2024-04-23T01:56:11.539613+0000","last_became_active":"2024-04-23T01:56:03.605401+0000","last_became_peered":"2024-04-23T01:56:03.605401+0000","last_unstale":"2024-04-23T01:56:11.539613+0000","last_undegraded":"2024-04-23T01:56:11.539613+0000","last_fullsized":"2024-04-23T01:56:11.539613+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":149,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.442901+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:56:11.442901+0000","last_peered":"2024-04-23T01:56:11.442901+0000","last_clean":"2024-04-23T01:56:11.442901+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:56:11.442901+0000","last_undegraded":"2024-04-23T01:56:11.442901+0000","last_fullsized":"2024-04-23T01:56:11.442901+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.c","version":"64'64","reported_seq":79,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.564141+0000","last_change":"2024-04-23T01:56:03.574136+0000","last_active":"2024-04-23T01:56:11.564141+0000","last_peered":"2024-04-23T01:56:11.564141+0000","last_clean":"2024-04-23T01:56:11.564141+0000","last_became_active":"2024-04-23T01:56:03.566135+0000","last_became_peered":"2024-04-23T01:56:03.566135+0000","last_unstale":"2024-04-23T01:56:11.564141+0000","last_undegraded":"2024-04-23T01:56:11.564141+0000","last_fullsized":"2024-04-23T01:56:11.564141+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-23T01:56:11.569818+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:56:11.569818+0000","last_peered":"2024-04-23T01:56:11.569818+0000","last_clean":"2024-04-23T01:56:11.569818+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:56:11.569818+0000","last_undegraded":"2024-04-23T01:56:11.569818+0000","last_fullsized":"2024-04-23T01:56:11.569818+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":8904704,"num_objects":138,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1087,"num_write_kb":8696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":13320192,"data_stored":26640384,"data_compressed":1372266,"data_compressed_allocated":13320192,"data_compressed_original":26640384,"omap_allocated":0,"internal_metadata":0},"log_size":1082,"ondisk_log_size":1082,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706328,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":103080,"kb_used_data":4064,"kb_used_omap":0,"kb_used_meta":99008,"kb_avail":94268760,"statfs":{"total":96636764160,"available":96531210240,"internally_reserved":0,"allocated":4161536,"data_stored":7884474,"data_compressed":395353,"data_compressed_allocated":3588096,"data_compressed_original":7667712,"omap_allocated":0,"internal_metadata":101384192},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706328,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":92512,"kb_used_data":3224,"kb_used_omap":0,"kb_used_meta":89280,"kb_avail":94279328,"statfs":{"total":96636764160,"available":96542031872,"internally_reserved":0,"allocated":3301376,"data_stored":6164154,"data_compressed":306741,"data_compressed_allocated":2727936,"data_compressed_original":5947392,"omap_allocated":0,"internal_metadata":91422720},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706328,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":114776,"kb_used_data":3280,"kb_used_omap":0,"kb_used_meta":111488,"kb_avail":94257064,"statfs":{"total":96636764160,"available":96519233536,"internally_reserved":0,"allocated":3358720,"data_stored":5798215,"data_compressed":288211,"data_compressed_allocated":2797568,"data_compressed_original":5595136,"omap_allocated":0,"internal_metadata":114163712},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706328,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":107696,"kb_used_data":4520,"kb_used_omap":0,"kb_used_meta":103168,"kb_avail":94264144,"statfs":{"total":96636764160,"available":96526483456,"internally_reserved":0,"allocated":4628480,"data_stored":8805223,"data_compressed":442633,"data_compressed_allocated":4046848,"data_compressed_original":8585216,"omap_allocated":0,"internal_metadata":105644032},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4022272,"data_stored":8044544,"data_compressed":414372,"data_compressed_allocated":4022272,"data_compressed_original":8044544,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2838528,"data_stored":5677056,"data_compressed":292441,"data_compressed_allocated":2838528,"data_compressed_original":5677056,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2813952,"data_stored":5627904,"data_compressed":289901,"data_compressed_allocated":2813952,"data_compressed_original":5627904,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3645440,"data_stored":7290880,"data_compressed":375552,"data_compressed_allocated":3645440,"data_compressed_original":7290880,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:56:16.475 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 12 16 1761 1745 1.13597 1.36719 0.0921487 0.109626 2024-04-23T01:56:17.476 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 13 16 1920 1904 1.14414 1.24219 0.0325937 0.108729 2024-04-23T01:56:17.637 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:56:17.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:17.757+0000 7f84f691d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:17.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:17.764+0000 7f84f691d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:18.476 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 14 16 2064 2048 1.14276 1.125 0.0604553 0.109006 2024-04-23T01:56:19.476 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 15 16 2220 2204 1.14782 1.21875 0.141058 0.108474 2024-04-23T01:56:20.002 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:56:20.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:20.142+0000 7fdbc535d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:20.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:20.151+0000 7fdbc535d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:20.476 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 16 16 2362 2346 1.14541 1.10938 0.0693755 0.108816 2024-04-23T01:56:20.506 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:56:20.506 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:56:20.506 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:56:20.506 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:56:20.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:20.629+0000 7f1583aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:20.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:20.640+0000 7f1583aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:20.686 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:56:20.686 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:56:20.686 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:56:20.687 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:56:20.687 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:56:20.687 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:56:20.687 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:56:20.687 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:56:20.687 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:56:20.687 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:56:20.687 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:56:20.687 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:56:20.690 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:56:20.690 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:56:20.691 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:56:20.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:20.804+0000 7fa20c39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:20.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:20.815+0000 7fa20c39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:20.833 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:56:20.833 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:56:20.833 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:56:20.833 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:56:20.833 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:56:20.833 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:56:20.833 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:56:20.833 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:56:20.833 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:56:20.833 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:56:20.834 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:56:20.834 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:56:20.836 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:56:20.836 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:56:20.837 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:56:20.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:20.955+0000 7fc1c6e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:20.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:20.966+0000 7fc1c6e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:21.005 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:56:21.005 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:56:21.005 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:56:21.005 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:56:21.005 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:56:21.005 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:56:21.005 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:56:21.005 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:56:21.005 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:56:21.005 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:56:21.005 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:56:21.005 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:56:21.008 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:56:21.008 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:56:21.008 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:56:21.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:21.136+0000 7f2d4866d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:21.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:21.147+0000 7f2d4866d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:21.194 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:56:21.194 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:56:21.194 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:56:21.194 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:56:21.194 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:56:21.194 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:56:21.194 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:56:21.194 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:56:21.194 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:56:21.194 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:56:21.194 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:56:21.195 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:56:21.197 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:56:21.198 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:56:21.476 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 17 16 2504 2488 1.14329 1.10938 0.0978892 0.1087 2024-04-23T01:56:22.345 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:56:22.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:22.465+0000 7fa30f4ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:22.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:22.472+0000 7fa30f4ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:22.476 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 18 16 2649 2633 1.1427 1.13281 0.112432 0.108924 2024-04-23T01:56:23.476 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 19 16 2768 2752 1.13149 0.929688 0.203816 0.110059 2024-04-23T01:56:24.476 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:56:24.476025+0000 min lat: 0.014953 max lat: 0.354791 avg lat: 0.109904 2024-04-23T01:56:24.476 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:56:24.476 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 20 16 2918 2902 1.1335 1.17188 0.0303723 0.109904 2024-04-23T01:56:24.633 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:56:24.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:24.745+0000 7f6fede2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:24.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:24.781+0000 7f6fede2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:25.476 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 21 16 3064 3048 1.13383 1.14062 0.117737 0.109866 2024-04-23T01:56:26.199 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:56:26.199 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:56:26.199 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:56:26.199 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:56:26.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:26.323+0000 7f78e4d54640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:26.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:26.335+0000 7f78e4d54640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:26.364 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:56:26.364 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:56:26.364 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:56:26.364 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:56:26.364 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:56:26.364 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:56:26.364 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:56:26.364 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:56:26.364 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:56:26.364 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:56:26.364 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:56:26.365 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:56:26.367 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:56:26.367 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:56:26.368 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:56:26.477 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 22 16 3204 3188 1.13201 1.09375 0.177751 0.109888 2024-04-23T01:56:26.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:26.487+0000 7fc1c759d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:26.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:26.496+0000 7fc1c759d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:26.515 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:56:26.515 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:56:26.515 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:56:26.515 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:56:26.515 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:56:26.515 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:56:26.516 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:56:26.516 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:56:26.516 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:56:26.516 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:56:26.516 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:56:26.516 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:56:26.519 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:56:26.519 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:56:26.519 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:56:26.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:26.640+0000 7fbaf9c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:26.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:26.650+0000 7fbaf9c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:26.674 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:56:26.675 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:56:26.675 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:56:26.675 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:56:26.675 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:56:26.675 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:56:26.675 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:56:26.675 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:56:26.675 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:56:26.675 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:56:26.675 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:56:26.675 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:56:26.678 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:56:26.678 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:56:26.678 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:56:26.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:26.791+0000 7fb627213640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:26.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:26.806+0000 7fb627213640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:26.826 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:56:26.826 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:56:26.826 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:56:26.826 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:56:26.826 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:56:26.826 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:56:26.826 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:56:26.826 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:56:26.826 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:56:26.826 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:56:26.826 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:56:26.826 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:56:26.829 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:56:26.829 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:56:26.978 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:56:27.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:27.116+0000 7fc84eead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:27.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:27.122+0000 7fc84eead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:27.476 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 23 16 3349 3333 1.13204 1.13281 0.0212334 0.110108 2024-04-23T01:56:28.476 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 24 16 3487 3471 1.12979 1.07812 0.0896773 0.110326 2024-04-23T01:56:29.322 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:56:29.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:29.435+0000 7fa66ca64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:29.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:29.442+0000 7fa66ca64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:29.477 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 25 16 3631 3615 1.12959 1.125 0.139396 0.11034 2024-04-23T01:56:30.477 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 26 16 3764 3748 1.12611 1.03906 0.166939 0.110734 2024-04-23T01:56:31.477 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 27 16 3913 3897 1.12751 1.16406 0.125717 0.110579 2024-04-23T01:56:31.597 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:56:31.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:31.730+0000 7ff6a1f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:31.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:31.738+0000 7ff6a1f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:31.830 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:56:31.830 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:56:31.831 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:56:31.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:31.948+0000 7fc14ec64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:31.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:31.957+0000 7fc14ec64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:32.229 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:56:32.229 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:56:32.241 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":407,"stamp":"2024-04-23T01:56:31.680842+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":26935840,"num_objects":406,"num_object_clones":0,"num_object_copies":1218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":3353,"num_write_kb":27622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3297,"ondisk_log_size":3297,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":451716,"kb_used_data":40420,"kb_used_omap":0,"kb_used_meta":411264,"kb_avail":377035644,"statfs":{"total":386547056640,"available":386084499456,"internally_reserved":0,"allocated":41390080,"data_stored":80282321,"data_compressed":4095723,"data_compressed_allocated":38944768,"data_compressed_original":79364096,"omap_allocated":0,"internal_metadata":421134336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5636096,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":688,"num_write_kb":5504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001873"},"pg_stats":[{"pgid":"9.7","version":"69'300","reported_seq":326,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T01:56:26.622906+0000","last_change":"2024-04-23T01:56:03.683930+0000","last_active":"2024-04-23T01:56:26.622906+0000","last_peered":"2024-04-23T01:56:26.622906+0000","last_clean":"2024-04-23T01:56:26.622906+0000","last_became_active":"2024-04-23T01:56:03.675554+0000","last_became_peered":"2024-04-23T01:56:03.675554+0000","last_unstale":"2024-04-23T01:56:26.622906+0000","last_undegraded":"2024-04-23T01:56:26.622906+0000","last_fullsized":"2024-04-23T01:56:26.622906+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2465792,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":301,"num_write_kb":2408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"70'247","reported_seq":274,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-23T01:56:26.282295+0000","last_change":"2024-04-23T01:56:03.552647+0000","last_active":"2024-04-23T01:56:26.282295+0000","last_peered":"2024-04-23T01:56:26.282295+0000","last_clean":"2024-04-23T01:56:26.282295+0000","last_became_active":"2024-04-23T01:56:03.541685+0000","last_became_peered":"2024-04-23T01:56:03.541685+0000","last_unstale":"2024-04-23T01:56:26.282295+0000","last_undegraded":"2024-04-23T01:56:26.282295+0000","last_fullsized":"2024-04-23T01:56:26.282295+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"69'207","reported_seq":233,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T01:56:25.776573+0000","last_change":"2024-04-23T01:56:03.515568+0000","last_active":"2024-04-23T01:56:25.776573+0000","last_peered":"2024-04-23T01:56:25.776573+0000","last_clean":"2024-04-23T01:56:25.776573+0000","last_became_active":"2024-04-23T01:56:03.510354+0000","last_became_peered":"2024-04-23T01:56:03.510354+0000","last_unstale":"2024-04-23T01:56:25.776573+0000","last_undegraded":"2024-04-23T01:56:25.776573+0000","last_fullsized":"2024-04-23T01:56:25.776573+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703936,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":208,"num_write_kb":1664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.4","version":"69'247","reported_seq":273,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T01:56:25.500810+0000","last_change":"2024-04-23T01:56:03.647359+0000","last_active":"2024-04-23T01:56:25.500810+0000","last_peered":"2024-04-23T01:56:25.500810+0000","last_clean":"2024-04-23T01:56:25.500810+0000","last_became_active":"2024-04-23T01:56:03.642592+0000","last_became_peered":"2024-04-23T01:56:03.642592+0000","last_unstale":"2024-04-23T01:56:25.500810+0000","last_undegraded":"2024-04-23T01:56:25.500810+0000","last_fullsized":"2024-04-23T01:56:25.500810+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"69'156","reported_seq":182,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T01:56:26.684248+0000","last_change":"2024-04-23T01:56:03.566150+0000","last_active":"2024-04-23T01:56:26.684248+0000","last_peered":"2024-04-23T01:56:26.684248+0000","last_clean":"2024-04-23T01:56:26.684248+0000","last_became_active":"2024-04-23T01:56:03.553086+0000","last_became_peered":"2024-04-23T01:56:03.553086+0000","last_unstale":"2024-04-23T01:56:26.684248+0000","last_undegraded":"2024-04-23T01:56:26.684248+0000","last_fullsized":"2024-04-23T01:56:26.684248+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1286144,"num_objects":20,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":157,"num_write_kb":1256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"70'119","reported_seq":146,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-23T01:56:24.650119+0000","last_change":"2024-04-23T01:56:03.605892+0000","last_active":"2024-04-23T01:56:24.650119+0000","last_peered":"2024-04-23T01:56:24.650119+0000","last_clean":"2024-04-23T01:56:24.650119+0000","last_became_active":"2024-04-23T01:56:03.601950+0000","last_became_peered":"2024-04-23T01:56:03.601950+0000","last_unstale":"2024-04-23T01:56:24.650119+0000","last_undegraded":"2024-04-23T01:56:24.650119+0000","last_fullsized":"2024-04-23T01:56:24.650119+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"69'119","reported_seq":145,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T01:56:24.104843+0000","last_change":"2024-04-23T01:56:03.618761+0000","last_active":"2024-04-23T01:56:24.104843+0000","last_peered":"2024-04-23T01:56:24.104843+0000","last_clean":"2024-04-23T01:56:24.104843+0000","last_became_active":"2024-04-23T01:56:03.602521+0000","last_became_peered":"2024-04-23T01:56:03.602521+0000","last_unstale":"2024-04-23T01:56:24.104843+0000","last_undegraded":"2024-04-23T01:56:24.104843+0000","last_fullsized":"2024-04-23T01:56:24.104843+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"69'223","reported_seq":249,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T01:56:26.235749+0000","last_change":"2024-04-23T01:56:03.544149+0000","last_active":"2024-04-23T01:56:26.235749+0000","last_peered":"2024-04-23T01:56:26.235749+0000","last_clean":"2024-04-23T01:56:26.235749+0000","last_became_active":"2024-04-23T01:56:03.539589+0000","last_became_peered":"2024-04-23T01:56:03.539589+0000","last_unstale":"2024-04-23T01:56:26.235749+0000","last_undegraded":"2024-04-23T01:56:26.235749+0000","last_fullsized":"2024-04-23T01:56:26.235749+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1835008,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":1792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"69'263","reported_seq":289,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T01:56:26.656549+0000","last_change":"2024-04-23T01:56:03.635647+0000","last_active":"2024-04-23T01:56:26.656549+0000","last_peered":"2024-04-23T01:56:26.656549+0000","last_clean":"2024-04-23T01:56:26.656549+0000","last_became_active":"2024-04-23T01:56:03.627392+0000","last_became_peered":"2024-04-23T01:56:03.627392+0000","last_unstale":"2024-04-23T01:56:26.656549+0000","last_undegraded":"2024-04-23T01:56:26.656549+0000","last_fullsized":"2024-04-23T01:56:26.656549+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162688,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":264,"num_write_kb":2112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":292,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T01:56:19.590636+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:56:19.590636+0000","last_peered":"2024-04-23T01:56:19.590636+0000","last_clean":"2024-04-23T01:56:19.590636+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:56:19.590636+0000","last_undegraded":"2024-04-23T01:56:19.590636+0000","last_fullsized":"2024-04-23T01:56:19.590636+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":162,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-23T01:56:24.277050+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:56:24.277050+0000","last_peered":"2024-04-23T01:56:24.277050+0000","last_clean":"2024-04-23T01:56:24.277050+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:56:24.277050+0000","last_undegraded":"2024-04-23T01:56:24.277050+0000","last_fullsized":"2024-04-23T01:56:24.277050+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.b","version":"69'175","reported_seq":201,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T01:56:25.895214+0000","last_change":"2024-04-23T01:56:03.687235+0000","last_active":"2024-04-23T01:56:25.895214+0000","last_peered":"2024-04-23T01:56:25.895214+0000","last_clean":"2024-04-23T01:56:25.895214+0000","last_became_active":"2024-04-23T01:56:03.683119+0000","last_became_peered":"2024-04-23T01:56:03.683119+0000","last_unstale":"2024-04-23T01:56:25.895214+0000","last_undegraded":"2024-04-23T01:56:25.895214+0000","last_fullsized":"2024-04-23T01:56:25.895214+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":153,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T01:56:19.619055+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:56:19.619055+0000","last_peered":"2024-04-23T01:56:19.619055+0000","last_clean":"2024-04-23T01:56:19.619055+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:56:19.619055+0000","last_undegraded":"2024-04-23T01:56:19.619055+0000","last_fullsized":"2024-04-23T01:56:19.619055+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"69'187","reported_seq":213,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T01:56:26.735593+0000","last_change":"2024-04-23T01:56:03.560440+0000","last_active":"2024-04-23T01:56:26.735593+0000","last_peered":"2024-04-23T01:56:26.735593+0000","last_clean":"2024-04-23T01:56:26.735593+0000","last_became_active":"2024-04-23T01:56:03.552840+0000","last_became_peered":"2024-04-23T01:56:03.552840+0000","last_unstale":"2024-04-23T01:56:26.735593+0000","last_undegraded":"2024-04-23T01:56:26.735593+0000","last_fullsized":"2024-04-23T01:56:26.735593+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1540096,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":188,"num_write_kb":1504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":162,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-23T01:56:24.258972+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:56:24.258972+0000","last_peered":"2024-04-23T01:56:24.258972+0000","last_clean":"2024-04-23T01:56:24.258972+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:56:24.258972+0000","last_undegraded":"2024-04-23T01:56:24.258972+0000","last_fullsized":"2024-04-23T01:56:24.258972+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"69'151","reported_seq":177,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T01:56:23.818628+0000","last_change":"2024-04-23T01:56:03.597772+0000","last_active":"2024-04-23T01:56:23.818628+0000","last_peered":"2024-04-23T01:56:23.818628+0000","last_clean":"2024-04-23T01:56:23.818628+0000","last_became_active":"2024-04-23T01:56:03.589730+0000","last_became_peered":"2024-04-23T01:56:03.589730+0000","last_unstale":"2024-04-23T01:56:23.818628+0000","last_undegraded":"2024-04-23T01:56:23.818628+0000","last_fullsized":"2024-04-23T01:56:23.818628+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":153,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T01:56:19.626436+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:56:19.626436+0000","last_peered":"2024-04-23T01:56:19.626436+0000","last_clean":"2024-04-23T01:56:19.626436+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:56:19.626436+0000","last_undegraded":"2024-04-23T01:56:19.626436+0000","last_fullsized":"2024-04-23T01:56:19.626436+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.f","version":"69'207","reported_seq":233,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T01:56:24.899448+0000","last_change":"2024-04-23T01:56:03.654933+0000","last_active":"2024-04-23T01:56:24.899448+0000","last_peered":"2024-04-23T01:56:24.899448+0000","last_clean":"2024-04-23T01:56:24.899448+0000","last_became_active":"2024-04-23T01:56:03.641185+0000","last_became_peered":"2024-04-23T01:56:03.641185+0000","last_unstale":"2024-04-23T01:56:24.899448+0000","last_undegraded":"2024-04-23T01:56:24.899448+0000","last_fullsized":"2024-04-23T01:56:24.899448+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703936,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":208,"num_write_kb":1664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":159,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T01:56:19.575557+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:56:19.575557+0000","last_peered":"2024-04-23T01:56:19.575557+0000","last_clean":"2024-04-23T01:56:19.575557+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:56:19.575557+0000","last_undegraded":"2024-04-23T01:56:19.575557+0000","last_fullsized":"2024-04-23T01:56:19.575557+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.e","version":"69'223","reported_seq":249,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T01:56:26.324247+0000","last_change":"2024-04-23T01:56:03.625197+0000","last_active":"2024-04-23T01:56:26.324247+0000","last_peered":"2024-04-23T01:56:26.324247+0000","last_clean":"2024-04-23T01:56:26.324247+0000","last_became_active":"2024-04-23T01:56:03.611498+0000","last_became_peered":"2024-04-23T01:56:03.611498+0000","last_unstale":"2024-04-23T01:56:26.324247+0000","last_undegraded":"2024-04-23T01:56:26.324247+0000","last_fullsized":"2024-04-23T01:56:26.324247+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1835008,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":1792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":159,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T01:56:19.612136+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:56:19.612136+0000","last_peered":"2024-04-23T01:56:19.612136+0000","last_clean":"2024-04-23T01:56:19.612136+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:56:19.612136+0000","last_undegraded":"2024-04-23T01:56:19.612136+0000","last_fullsized":"2024-04-23T01:56:19.612136+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"69'199","reported_seq":225,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T01:56:25.638208+0000","last_change":"2024-04-23T01:56:03.612020+0000","last_active":"2024-04-23T01:56:25.638208+0000","last_peered":"2024-04-23T01:56:25.638208+0000","last_clean":"2024-04-23T01:56:25.638208+0000","last_became_active":"2024-04-23T01:56:03.605401+0000","last_became_peered":"2024-04-23T01:56:03.605401+0000","last_unstale":"2024-04-23T01:56:25.638208+0000","last_undegraded":"2024-04-23T01:56:25.638208+0000","last_fullsized":"2024-04-23T01:56:25.638208+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":200,"num_write_kb":1600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":159,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T01:56:19.575714+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:56:19.575714+0000","last_peered":"2024-04-23T01:56:19.575714+0000","last_clean":"2024-04-23T01:56:19.575714+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:56:19.575714+0000","last_undegraded":"2024-04-23T01:56:19.575714+0000","last_fullsized":"2024-04-23T01:56:19.575714+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.c","version":"70'177","reported_seq":204,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-23T01:56:26.638449+0000","last_change":"2024-04-23T01:56:03.574136+0000","last_active":"2024-04-23T01:56:26.638449+0000","last_peered":"2024-04-23T01:56:26.638449+0000","last_clean":"2024-04-23T01:56:26.638449+0000","last_became_active":"2024-04-23T01:56:03.566135+0000","last_became_peered":"2024-04-23T01:56:03.566135+0000","last_unstale":"2024-04-23T01:56:26.638449+0000","last_undegraded":"2024-04-23T01:56:26.638449+0000","last_fullsized":"2024-04-23T01:56:26.638449+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1458176,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":178,"num_write_kb":1424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":162,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-23T01:56:24.254689+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:56:24.254689+0000","last_peered":"2024-04-23T01:56:24.254689+0000","last_clean":"2024-04-23T01:56:24.254689+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:56:24.254689+0000","last_undegraded":"2024-04-23T01:56:24.254689+0000","last_fullsized":"2024-04-23T01:56:24.254689+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":26345472,"num_objects":404,"num_object_clones":0,"num_object_copies":1212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3216,"num_write_kb":25728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":39493632,"data_stored":78987264,"data_compressed":4075236,"data_compressed_allocated":39493632,"data_compressed_original":78987264,"omap_allocated":0,"internal_metadata":0},"log_size":3200,"ondisk_log_size":3200,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706343,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":118468,"kb_used_data":11132,"kb_used_omap":0,"kb_used_meta":107328,"kb_avail":94253372,"statfs":{"total":96636764160,"available":96515452928,"internally_reserved":0,"allocated":11399168,"data_stored":22294035,"data_compressed":1138564,"data_compressed_allocated":10784768,"data_compressed_original":22061056,"omap_allocated":0,"internal_metadata":109903872},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706343,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":101928,"kb_used_data":8480,"kb_used_omap":0,"kb_used_meta":93440,"kb_avail":94269912,"statfs":{"total":96636764160,"available":96532389888,"internally_reserved":0,"allocated":8683520,"data_stored":16862739,"data_compressed":858333,"data_compressed_allocated":8069120,"data_compressed_original":16629760,"omap_allocated":0,"internal_metadata":95682560},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706343,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":107820,"kb_used_data":8804,"kb_used_omap":0,"kb_used_meta":99008,"kb_avail":94264020,"statfs":{"total":96636764160,"available":96526356480,"internally_reserved":0,"allocated":9015296,"data_stored":17045656,"data_compressed":868144,"data_compressed_allocated":8413184,"data_compressed_original":16826368,"omap_allocated":0,"internal_metadata":101384192},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706342,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":123500,"kb_used_data":12004,"kb_used_omap":0,"kb_used_meta":111488,"kb_avail":94248340,"statfs":{"total":96636764160,"available":96510300160,"internally_reserved":0,"allocated":12292096,"data_stored":24079891,"data_compressed":1230682,"data_compressed_allocated":11677696,"data_compressed_original":23846912,"omap_allocated":0,"internal_metadata":114163712},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12115968,"data_stored":24231936,"data_compressed":1250220,"data_compressed_allocated":12115968,"data_compressed_original":24231936,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8450048,"data_stored":16900096,"data_compressed":871951,"data_compressed_allocated":8450048,"data_compressed_original":16900096,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8069120,"data_stored":16138240,"data_compressed":832610,"data_compressed_allocated":8069120,"data_compressed_original":16138240,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10858496,"data_stored":21716992,"data_compressed":1120455,"data_compressed_allocated":10858496,"data_compressed_original":21716992,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:56:32.477 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 28 16 4044 4028 1.12379 1.02344 0.0251235 0.110928 2024-04-23T01:56:33.477 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 29 16 4196 4180 1.12598 1.1875 0.191663 0.110778 2024-04-23T01:56:33.934 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:56:34.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:34.068+0000 7f840df9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:34.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:34.074+0000 7f840df9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:34.477 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 30 16 4331 4315 1.1236 1.05469 0.175069 0.110945 2024-04-23T01:56:35.477 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 31 16 4487 4471 1.12667 1.21875 0.15408 0.110736 2024-04-23T01:56:36.309 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:56:36.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:36.429+0000 7f8e2e006640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:36.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:36.435+0000 7f8e2e006640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:36.477 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 32 16 4615 4599 1.12271 1 0.171429 0.1111 2024-04-23T01:56:37.243 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:56:37.243 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:56:37.243 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:56:37.243 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:56:37.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:37.377+0000 7f4b4186d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:37.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:37.386+0000 7f4b4186d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:37.423 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:56:37.423 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:56:37.423 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:56:37.423 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:56:37.423 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:56:37.423 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:56:37.423 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:56:37.423 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:56:37.423 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:56:37.423 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:56:37.423 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:56:37.423 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:56:37.426 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:56:37.426 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:56:37.426 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:56:37.477 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 33 16 4756 4740 1.12206 1.10156 0.101222 0.111187 2024-04-23T01:56:37.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:37.541+0000 7f217e39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:37.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:37.553+0000 7f217e39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:37.581 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:56:37.582 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:56:37.582 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:56:37.582 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:56:37.582 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:56:37.582 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:56:37.582 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:56:37.582 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:56:37.582 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:56:37.582 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:56:37.582 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:56:37.582 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:56:37.585 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:56:37.585 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:56:37.585 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:56:37.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:37.695+0000 7fa70e0ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:37.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:37.706+0000 7fa70e0ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:37.738 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:56:37.738 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:56:37.738 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:56:37.739 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:56:37.739 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:56:37.739 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:56:37.739 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:56:37.739 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:56:37.739 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:56:37.739 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:56:37.739 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:56:37.739 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:56:37.742 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:56:37.743 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:56:37.743 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:56:37.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:37.853+0000 7f244586d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:37.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:37.864+0000 7f244586d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:37.947 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:56:37.947 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:56:37.947 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:56:37.947 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:56:37.948 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:56:37.948 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:56:37.948 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:56:37.948 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:56:37.948 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:56:37.948 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:56:37.948 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:56:37.948 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:56:37.951 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:56:37.951 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:56:38.477 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 34 16 4903 4887 1.12284 1.14844 0.160898 0.111119 2024-04-23T01:56:38.622 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:56:38.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:38.749+0000 7f5241bff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:38.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:38.755+0000 7f5241bff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:39.478 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 35 16 5041 5025 1.12156 1.07812 0.126875 0.11121 2024-04-23T01:56:40.478 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 36 16 5185 5169 1.12165 1.125 0.0465972 0.111207 2024-04-23T01:56:40.952 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:56:41.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:41.063+0000 7fcccdeed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:41.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:41.070+0000 7fcccdeed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:41.478 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 37 16 5315 5299 1.11878 1.01562 0.163392 0.111478 2024-04-23T01:56:42.478 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 38 16 5451 5435 1.1173 1.0625 0.128135 0.111648 2024-04-23T01:56:42.952 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:56:42.952 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:56:42.953 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:56:43.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:43.063+0000 7f4c01d1d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:43.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:43.071+0000 7f4c01d1d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:43.187 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:56:43.278 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:56:43.279 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:56:43.293 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2024-04-23T01:56:41.683125+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":40034848,"num_objects":605,"num_object_clones":0,"num_object_copies":1815,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":4952,"num_write_kb":40414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4899,"ondisk_log_size":4899,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":476852,"kb_used_data":59988,"kb_used_omap":0,"kb_used_meta":416832,"kb_avail":377010508,"statfs":{"total":386547056640,"available":386058760192,"internally_reserved":0,"allocated":61427712,"data_stored":120212968,"data_compressed":6155733,"data_compressed_allocated":58892288,"data_compressed_original":119259136,"omap_allocated":0,"internal_metadata":426835968},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13099008,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1599,"num_write_kb":12792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002282"},"pg_stats":[{"pgid":"9.7","version":"72'487","reported_seq":522,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:41.465145+0000","last_change":"2024-04-23T01:56:03.683930+0000","last_active":"2024-04-23T01:56:41.465145+0000","last_peered":"2024-04-23T01:56:41.465145+0000","last_clean":"2024-04-23T01:56:41.465145+0000","last_became_active":"2024-04-23T01:56:03.675554+0000","last_became_peered":"2024-04-23T01:56:03.675554+0000","last_unstale":"2024-04-23T01:56:41.465145+0000","last_undegraded":"2024-04-23T01:56:41.465145+0000","last_fullsized":"2024-04-23T01:56:41.465145+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3997696,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":488,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"72'319","reported_seq":352,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:36.468949+0000","last_change":"2024-04-23T01:56:03.552647+0000","last_active":"2024-04-23T01:56:36.468949+0000","last_peered":"2024-04-23T01:56:36.468949+0000","last_clean":"2024-04-23T01:56:36.468949+0000","last_became_active":"2024-04-23T01:56:03.541685+0000","last_became_peered":"2024-04-23T01:56:03.541685+0000","last_unstale":"2024-04-23T01:56:36.468949+0000","last_undegraded":"2024-04-23T01:56:36.468949+0000","last_fullsized":"2024-04-23T01:56:36.468949+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":320,"num_write_kb":2560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"72'287","reported_seq":322,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:36.133388+0000","last_change":"2024-04-23T01:56:03.515568+0000","last_active":"2024-04-23T01:56:36.133388+0000","last_peered":"2024-04-23T01:56:36.133388+0000","last_clean":"2024-04-23T01:56:36.133388+0000","last_became_active":"2024-04-23T01:56:03.510354+0000","last_became_peered":"2024-04-23T01:56:03.510354+0000","last_unstale":"2024-04-23T01:56:36.133388+0000","last_undegraded":"2024-04-23T01:56:36.133388+0000","last_fullsized":"2024-04-23T01:56:36.133388+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.4","version":"72'319","reported_seq":354,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:36.104874+0000","last_change":"2024-04-23T01:56:03.647359+0000","last_active":"2024-04-23T01:56:36.104874+0000","last_peered":"2024-04-23T01:56:36.104874+0000","last_clean":"2024-04-23T01:56:36.104874+0000","last_became_active":"2024-04-23T01:56:03.642592+0000","last_became_peered":"2024-04-23T01:56:03.642592+0000","last_unstale":"2024-04-23T01:56:36.104874+0000","last_undegraded":"2024-04-23T01:56:36.104874+0000","last_fullsized":"2024-04-23T01:56:36.104874+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":320,"num_write_kb":2560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"71'232","reported_seq":266,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:34.264826+0000","last_change":"2024-04-23T01:56:03.566150+0000","last_active":"2024-04-23T01:56:34.264826+0000","last_peered":"2024-04-23T01:56:34.264826+0000","last_clean":"2024-04-23T01:56:34.264826+0000","last_became_active":"2024-04-23T01:56:03.553086+0000","last_became_peered":"2024-04-23T01:56:03.553086+0000","last_unstale":"2024-04-23T01:56:34.264826+0000","last_undegraded":"2024-04-23T01:56:34.264826+0000","last_fullsized":"2024-04-23T01:56:34.264826+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1900544,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":232,"num_write_kb":1856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"71'216","reported_seq":248,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:35.579390+0000","last_change":"2024-04-23T01:56:03.605892+0000","last_active":"2024-04-23T01:56:35.579390+0000","last_peered":"2024-04-23T01:56:35.579390+0000","last_clean":"2024-04-23T01:56:35.579390+0000","last_became_active":"2024-04-23T01:56:03.601950+0000","last_became_peered":"2024-04-23T01:56:03.601950+0000","last_unstale":"2024-04-23T01:56:35.579390+0000","last_undegraded":"2024-04-23T01:56:35.579390+0000","last_fullsized":"2024-04-23T01:56:35.579390+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1769472,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":216,"num_write_kb":1728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"72'199","reported_seq":234,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:37.243591+0000","last_change":"2024-04-23T01:56:03.618761+0000","last_active":"2024-04-23T01:56:37.243591+0000","last_peered":"2024-04-23T01:56:37.243591+0000","last_clean":"2024-04-23T01:56:37.243591+0000","last_became_active":"2024-04-23T01:56:03.602521+0000","last_became_peered":"2024-04-23T01:56:03.602521+0000","last_unstale":"2024-04-23T01:56:37.243591+0000","last_undegraded":"2024-04-23T01:56:37.243591+0000","last_fullsized":"2024-04-23T01:56:37.243591+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":200,"num_write_kb":1600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"71'320","reported_seq":354,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:35.544302+0000","last_change":"2024-04-23T01:56:03.544149+0000","last_active":"2024-04-23T01:56:35.544302+0000","last_peered":"2024-04-23T01:56:35.544302+0000","last_clean":"2024-04-23T01:56:35.544302+0000","last_became_active":"2024-04-23T01:56:03.539589+0000","last_became_peered":"2024-04-23T01:56:03.539589+0000","last_unstale":"2024-04-23T01:56:35.544302+0000","last_undegraded":"2024-04-23T01:56:35.544302+0000","last_fullsized":"2024-04-23T01:56:35.544302+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":320,"num_write_kb":2560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"72'375","reported_seq":410,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:36.350046+0000","last_change":"2024-04-23T01:56:03.635647+0000","last_active":"2024-04-23T01:56:36.350046+0000","last_peered":"2024-04-23T01:56:36.350046+0000","last_clean":"2024-04-23T01:56:36.350046+0000","last_became_active":"2024-04-23T01:56:03.627392+0000","last_became_peered":"2024-04-23T01:56:03.627392+0000","last_unstale":"2024-04-23T01:56:36.350046+0000","last_undegraded":"2024-04-23T01:56:36.350046+0000","last_fullsized":"2024-04-23T01:56:36.350046+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":3008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":301,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:34.244954+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:56:34.244954+0000","last_peered":"2024-04-23T01:56:34.244954+0000","last_clean":"2024-04-23T01:56:34.244954+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:56:34.244954+0000","last_undegraded":"2024-04-23T01:56:34.244954+0000","last_fullsized":"2024-04-23T01:56:34.244954+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":168,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:35.582167+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:56:35.582167+0000","last_peered":"2024-04-23T01:56:35.582167+0000","last_clean":"2024-04-23T01:56:35.582167+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:56:35.582167+0000","last_undegraded":"2024-04-23T01:56:35.582167+0000","last_fullsized":"2024-04-23T01:56:35.582167+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.b","version":"72'295","reported_seq":330,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:40.724259+0000","last_change":"2024-04-23T01:56:03.687235+0000","last_active":"2024-04-23T01:56:40.724259+0000","last_peered":"2024-04-23T01:56:40.724259+0000","last_clean":"2024-04-23T01:56:40.724259+0000","last_became_active":"2024-04-23T01:56:03.683119+0000","last_became_peered":"2024-04-23T01:56:03.683119+0000","last_unstale":"2024-04-23T01:56:40.724259+0000","last_undegraded":"2024-04-23T01:56:40.724259+0000","last_fullsized":"2024-04-23T01:56:40.724259+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2424832,"num_objects":37,"num_object_clones":0,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":296,"num_write_kb":2368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":162,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:35.619522+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:56:35.619522+0000","last_peered":"2024-04-23T01:56:35.619522+0000","last_clean":"2024-04-23T01:56:35.619522+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:56:35.619522+0000","last_undegraded":"2024-04-23T01:56:35.619522+0000","last_fullsized":"2024-04-23T01:56:35.619522+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"72'263","reported_seq":298,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:35.716941+0000","last_change":"2024-04-23T01:56:03.560440+0000","last_active":"2024-04-23T01:56:35.716941+0000","last_peered":"2024-04-23T01:56:35.716941+0000","last_clean":"2024-04-23T01:56:35.716941+0000","last_became_active":"2024-04-23T01:56:03.552840+0000","last_became_peered":"2024-04-23T01:56:03.552840+0000","last_unstale":"2024-04-23T01:56:35.716941+0000","last_undegraded":"2024-04-23T01:56:35.716941+0000","last_fullsized":"2024-04-23T01:56:35.716941+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162688,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":264,"num_write_kb":2112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":168,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:35.560551+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:56:35.560551+0000","last_peered":"2024-04-23T01:56:35.560551+0000","last_clean":"2024-04-23T01:56:35.560551+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:56:35.560551+0000","last_undegraded":"2024-04-23T01:56:35.560551+0000","last_fullsized":"2024-04-23T01:56:35.560551+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"72'279","reported_seq":314,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:36.457583+0000","last_change":"2024-04-23T01:56:03.597772+0000","last_active":"2024-04-23T01:56:36.457583+0000","last_peered":"2024-04-23T01:56:36.457583+0000","last_clean":"2024-04-23T01:56:36.457583+0000","last_became_active":"2024-04-23T01:56:03.589730+0000","last_became_peered":"2024-04-23T01:56:03.589730+0000","last_unstale":"2024-04-23T01:56:36.457583+0000","last_undegraded":"2024-04-23T01:56:36.457583+0000","last_fullsized":"2024-04-23T01:56:36.457583+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2293760,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":280,"num_write_kb":2240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":162,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:35.632182+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:56:35.632182+0000","last_peered":"2024-04-23T01:56:35.632182+0000","last_clean":"2024-04-23T01:56:35.632182+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:56:35.632182+0000","last_undegraded":"2024-04-23T01:56:35.632182+0000","last_fullsized":"2024-04-23T01:56:35.632182+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.f","version":"72'327","reported_seq":362,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:41.330852+0000","last_change":"2024-04-23T01:56:03.654933+0000","last_active":"2024-04-23T01:56:41.330852+0000","last_peered":"2024-04-23T01:56:41.330852+0000","last_clean":"2024-04-23T01:56:41.330852+0000","last_became_active":"2024-04-23T01:56:03.641185+0000","last_became_peered":"2024-04-23T01:56:03.641185+0000","last_unstale":"2024-04-23T01:56:41.330852+0000","last_undegraded":"2024-04-23T01:56:41.330852+0000","last_fullsized":"2024-04-23T01:56:41.330852+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2686976,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":328,"num_write_kb":2624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":168,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:35.544714+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:56:35.544714+0000","last_peered":"2024-04-23T01:56:35.544714+0000","last_clean":"2024-04-23T01:56:35.544714+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:56:35.544714+0000","last_undegraded":"2024-04-23T01:56:35.544714+0000","last_fullsized":"2024-04-23T01:56:35.544714+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.e","version":"72'335","reported_seq":370,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:40.464655+0000","last_change":"2024-04-23T01:56:03.625197+0000","last_active":"2024-04-23T01:56:40.464655+0000","last_peered":"2024-04-23T01:56:40.464655+0000","last_clean":"2024-04-23T01:56:40.464655+0000","last_became_active":"2024-04-23T01:56:03.611498+0000","last_became_peered":"2024-04-23T01:56:03.611498+0000","last_unstale":"2024-04-23T01:56:40.464655+0000","last_undegraded":"2024-04-23T01:56:40.464655+0000","last_fullsized":"2024-04-23T01:56:40.464655+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":168,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:34.250754+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:56:34.250754+0000","last_peered":"2024-04-23T01:56:34.250754+0000","last_clean":"2024-04-23T01:56:34.250754+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:56:34.250754+0000","last_undegraded":"2024-04-23T01:56:34.250754+0000","last_fullsized":"2024-04-23T01:56:34.250754+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"72'293","reported_seq":328,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:36.721925+0000","last_change":"2024-04-23T01:56:03.612020+0000","last_active":"2024-04-23T01:56:36.721925+0000","last_peered":"2024-04-23T01:56:36.721925+0000","last_clean":"2024-04-23T01:56:36.721925+0000","last_became_active":"2024-04-23T01:56:03.605401+0000","last_became_peered":"2024-04-23T01:56:03.605401+0000","last_unstale":"2024-04-23T01:56:36.721925+0000","last_undegraded":"2024-04-23T01:56:36.721925+0000","last_fullsized":"2024-04-23T01:56:36.721925+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2408448,"num_objects":37,"num_object_clones":0,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":294,"num_write_kb":2352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":168,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:35.552765+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:56:35.552765+0000","last_peered":"2024-04-23T01:56:35.552765+0000","last_clean":"2024-04-23T01:56:35.552765+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:56:35.552765+0000","last_undegraded":"2024-04-23T01:56:35.552765+0000","last_fullsized":"2024-04-23T01:56:35.552765+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.c","version":"71'256","reported_seq":289,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:35.565586+0000","last_change":"2024-04-23T01:56:03.574136+0000","last_active":"2024-04-23T01:56:35.565586+0000","last_peered":"2024-04-23T01:56:35.565586+0000","last_clean":"2024-04-23T01:56:35.565586+0000","last_became_active":"2024-04-23T01:56:03.566135+0000","last_became_peered":"2024-04-23T01:56:03.566135+0000","last_unstale":"2024-04-23T01:56:35.565586+0000","last_undegraded":"2024-04-23T01:56:35.565586+0000","last_fullsized":"2024-04-23T01:56:35.565586+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2105344,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":257,"num_write_kb":2056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":168,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:35.569657+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:56:35.569657+0000","last_peered":"2024-04-23T01:56:35.569657+0000","last_clean":"2024-04-23T01:56:35.569657+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:56:35.569657+0000","last_undegraded":"2024-04-23T01:56:35.569657+0000","last_fullsized":"2024-04-23T01:56:35.569657+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":39444480,"num_objects":603,"num_object_clones":0,"num_object_copies":1809,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4815,"num_write_kb":38520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":59486208,"data_stored":118972416,"data_compressed":6139899,"data_compressed_allocated":59486208,"data_compressed_original":118972416,"omap_allocated":0,"internal_metadata":0},"log_size":4802,"ondisk_log_size":4802,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706353,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":127192,"kb_used_data":15696,"kb_used_omap":0,"kb_used_meta":111488,"kb_avail":94244648,"statfs":{"total":96636764160,"available":96506519552,"internally_reserved":0,"allocated":16072704,"data_stored":31601666,"data_compressed":1618669,"data_compressed_allocated":15433728,"data_compressed_original":31358976,"omap_allocated":0,"internal_metadata":114163712},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706353,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":110968,"kb_used_data":11952,"kb_used_omap":0,"kb_used_meta":99008,"kb_avail":94260872,"statfs":{"total":96636764160,"available":96523132928,"internally_reserved":0,"allocated":12238848,"data_stored":23933954,"data_compressed":1222959,"data_compressed_allocated":11599872,"data_compressed_original":23691264,"omap_allocated":0,"internal_metadata":101384192},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706353,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":115568,"kb_used_data":12392,"kb_used_omap":0,"kb_used_meta":103168,"kb_avail":94256272,"statfs":{"total":96636764160,"available":96518422528,"internally_reserved":0,"allocated":12689408,"data_stored":24367586,"data_compressed":1245883,"data_compressed_allocated":12070912,"data_compressed_original":24141824,"omap_allocated":0,"internal_metadata":105644032},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706357,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":123124,"kb_used_data":19948,"kb_used_omap":0,"kb_used_meta":103168,"kb_avail":94248716,"statfs":{"total":96636764160,"available":96510685184,"internally_reserved":0,"allocated":20426752,"data_stored":40309762,"data_compressed":2068222,"data_compressed_allocated":19787776,"data_compressed_original":40067072,"omap_allocated":0,"internal_metadata":105644032},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20209664,"data_stored":40419328,"data_compressed":2086068,"data_compressed_allocated":20209664,"data_compressed_original":40419328,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12099584,"data_stored":24199168,"data_compressed":1248844,"data_compressed_allocated":12099584,"data_compressed_original":24199168,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":11694080,"data_stored":23388160,"data_compressed":1206965,"data_compressed_allocated":11694080,"data_compressed_original":23388160,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":15482880,"data_stored":30965760,"data_compressed":1598022,"data_compressed_allocated":15482880,"data_compressed_original":30965760,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:56:43.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:43.305+0000 7fa12f66d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:43.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:43.312+0000 7fa12f66d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:43.478 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 39 16 5576 5560 1.11369 0.976562 0.187866 0.111945 2024-04-23T01:56:44.478 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:56:44.477853+0000 min lat: 0.014953 max lat: 0.364968 avg lat: 0.11226 2024-04-23T01:56:44.478 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:56:44.478 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 40 16 5707 5691 1.11143 1.02344 0.0168392 0.11226 2024-04-23T01:56:45.447 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:56:45.478 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 41 16 5845 5829 1.11061 1.07812 0.14949 0.112354 2024-04-23T01:56:45.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:45.570+0000 7fc7f086d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:45.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:45.578+0000 7fc7f086d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:46.478 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 42 16 5975 5959 1.10835 1.01562 0.029925 0.11262 2024-04-23T01:56:47.478 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 43 16 6107 6091 1.10655 1.03125 0.0158947 0.112755 2024-04-23T01:56:47.891 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:56:48.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:48.029+0000 7ff43102d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:48.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:48.038+0000 7ff43102d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:48.295 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:56:48.426 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:56:48.426 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:56:48.427 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:56:48.478 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 44 16 6239 6223 1.10484 1.03125 0.150807 0.112906 2024-04-23T01:56:48.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:48.549+0000 7f1e3875d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:48.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:48.560+0000 7f1e3875d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:48.578 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:56:48.578 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:56:48.579 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:56:48.579 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:56:48.579 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:56:48.579 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:56:48.579 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:56:48.579 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:56:48.579 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:56:48.579 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:56:48.579 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:56:48.579 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:56:48.581 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:56:48.582 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:56:48.582 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:56:48.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:48.702+0000 7fc04699d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:48.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:48.712+0000 7fc04699d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:48.741 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:56:48.741 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:56:48.741 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:56:48.741 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:56:48.741 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:56:48.741 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:56:48.741 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:56:48.741 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:56:48.741 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:56:48.742 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:56:48.742 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:56:48.742 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:56:48.745 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:56:48.746 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:56:48.746 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:56:48.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:48.885+0000 7f8d945ff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:48.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:48.899+0000 7f8d9339d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:48.929 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:56:48.929 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:56:48.929 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:56:48.929 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:56:48.929 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:56:48.930 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:56:48.930 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:56:48.930 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:56:48.930 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:56:48.930 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:56:48.930 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:56:48.930 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:56:48.933 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:56:48.933 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:56:48.934 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:56:49.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:49.069+0000 7fe21e79d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:49.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:49.080+0000 7fe21e79d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:49.129 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:56:49.130 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:56:49.130 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:56:49.130 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:56:49.130 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:56:49.130 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:56:49.130 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:56:49.130 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:56:49.130 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:56:49.130 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:56:49.130 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:56:49.130 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:56:49.133 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:56:49.133 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:56:49.479 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 45 16 6382 6366 1.10511 1.11719 0.0369631 0.112903 2024-04-23T01:56:50.266 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:56:50.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:50.389+0000 7fc1e10ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:50.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:50.396+0000 7fc1e10ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:50.479 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 46 16 6517 6501 1.10401 1.05469 0.148369 0.113065 2024-04-23T01:56:51.479 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 47 16 6659 6643 1.10413 1.10938 0.115593 0.112963 2024-04-23T01:56:52.479 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 48 16 6797 6781 1.10358 1.07812 0.0570662 0.113104 2024-04-23T01:56:52.617 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:56:52.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:52.745+0000 7fc1f7f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:52.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:52.752+0000 7fc1f7f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:53.479 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 49 16 6927 6911 1.10178 1.01562 0.144019 0.113271 2024-04-23T01:56:54.134 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:56:54.134 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:56:54.134 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:56:54.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:54.269+0000 7febbd79d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:54.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:54.279+0000 7febbd79d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:54.479 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 50 16 7046 7030 1.09834 0.929688 0.0361841 0.113593 2024-04-23T01:56:54.545 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:56:54.545 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:56:54.564 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":419,"stamp":"2024-04-23T01:56:53.685706+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":55263776,"num_objects":837,"num_object_clones":0,"num_object_copies":2511,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":6811,"num_write_kb":55286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6755,"ondisk_log_size":6755,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":503000,"kb_used_data":81972,"kb_used_omap":0,"kb_used_meta":420992,"kb_avail":376984360,"statfs":{"total":386547056640,"available":386031984640,"internally_reserved":0,"allocated":83939328,"data_stored":165236200,"data_compressed":8480541,"data_compressed_allocated":81403904,"data_compressed_original":164282368,"omap_allocated":0,"internal_metadata":431095808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15228928,"num_objects":232,"num_object_clones":0,"num_object_copies":696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1859,"num_write_kb":14872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002580"},"pg_stats":[{"pgid":"9.7","version":"72'575","reported_seq":610,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:51.555131+0000","last_change":"2024-04-23T01:56:03.683930+0000","last_active":"2024-04-23T01:56:51.555131+0000","last_peered":"2024-04-23T01:56:51.555131+0000","last_clean":"2024-04-23T01:56:51.555131+0000","last_became_active":"2024-04-23T01:56:03.675554+0000","last_became_peered":"2024-04-23T01:56:03.675554+0000","last_unstale":"2024-04-23T01:56:51.555131+0000","last_undegraded":"2024-04-23T01:56:51.555131+0000","last_fullsized":"2024-04-23T01:56:51.555131+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718592,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":576,"num_write_kb":4608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"72'463","reported_seq":496,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:50.694075+0000","last_change":"2024-04-23T01:56:03.552647+0000","last_active":"2024-04-23T01:56:50.694075+0000","last_peered":"2024-04-23T01:56:50.694075+0000","last_clean":"2024-04-23T01:56:50.694075+0000","last_became_active":"2024-04-23T01:56:03.541685+0000","last_became_peered":"2024-04-23T01:56:03.541685+0000","last_unstale":"2024-04-23T01:56:50.694075+0000","last_undegraded":"2024-04-23T01:56:50.694075+0000","last_fullsized":"2024-04-23T01:56:50.694075+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":464,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"72'415","reported_seq":450,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:50.395504+0000","last_change":"2024-04-23T01:56:03.515568+0000","last_active":"2024-04-23T01:56:50.395504+0000","last_peered":"2024-04-23T01:56:50.395504+0000","last_clean":"2024-04-23T01:56:50.395504+0000","last_became_active":"2024-04-23T01:56:03.510354+0000","last_became_peered":"2024-04-23T01:56:03.510354+0000","last_unstale":"2024-04-23T01:56:50.395504+0000","last_undegraded":"2024-04-23T01:56:50.395504+0000","last_fullsized":"2024-04-23T01:56:50.395504+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3407872,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":3328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.4","version":"72'439","reported_seq":474,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:51.677172+0000","last_change":"2024-04-23T01:56:03.647359+0000","last_active":"2024-04-23T01:56:51.677172+0000","last_peered":"2024-04-23T01:56:51.677172+0000","last_clean":"2024-04-23T01:56:51.677172+0000","last_became_active":"2024-04-23T01:56:03.642592+0000","last_became_peered":"2024-04-23T01:56:03.642592+0000","last_unstale":"2024-04-23T01:56:51.677172+0000","last_undegraded":"2024-04-23T01:56:51.677172+0000","last_fullsized":"2024-04-23T01:56:51.677172+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3604480,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":440,"num_write_kb":3520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"72'399","reported_seq":434,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:50.836552+0000","last_change":"2024-04-23T01:56:03.566150+0000","last_active":"2024-04-23T01:56:50.836552+0000","last_peered":"2024-04-23T01:56:50.836552+0000","last_clean":"2024-04-23T01:56:50.836552+0000","last_became_active":"2024-04-23T01:56:03.553086+0000","last_became_peered":"2024-04-23T01:56:03.553086+0000","last_unstale":"2024-04-23T01:56:50.836552+0000","last_undegraded":"2024-04-23T01:56:50.836552+0000","last_fullsized":"2024-04-23T01:56:50.836552+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3276800,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":400,"num_write_kb":3200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"72'367","reported_seq":400,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:50.836153+0000","last_change":"2024-04-23T01:56:03.605892+0000","last_active":"2024-04-23T01:56:50.836153+0000","last_peered":"2024-04-23T01:56:50.836153+0000","last_clean":"2024-04-23T01:56:50.836153+0000","last_became_active":"2024-04-23T01:56:03.601950+0000","last_became_peered":"2024-04-23T01:56:03.601950+0000","last_unstale":"2024-04-23T01:56:50.836153+0000","last_undegraded":"2024-04-23T01:56:50.836153+0000","last_fullsized":"2024-04-23T01:56:50.836153+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3014656,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":368,"num_write_kb":2944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"72'282","reported_seq":317,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:51.627833+0000","last_change":"2024-04-23T01:56:03.618761+0000","last_active":"2024-04-23T01:56:51.627833+0000","last_peered":"2024-04-23T01:56:51.627833+0000","last_clean":"2024-04-23T01:56:51.627833+0000","last_became_active":"2024-04-23T01:56:03.602521+0000","last_became_peered":"2024-04-23T01:56:03.602521+0000","last_unstale":"2024-04-23T01:56:51.627833+0000","last_undegraded":"2024-04-23T01:56:51.627833+0000","last_fullsized":"2024-04-23T01:56:51.627833+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2318336,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":283,"num_write_kb":2264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"72'431","reported_seq":466,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:50.076656+0000","last_change":"2024-04-23T01:56:03.544149+0000","last_active":"2024-04-23T01:56:50.076656+0000","last_peered":"2024-04-23T01:56:50.076656+0000","last_clean":"2024-04-23T01:56:50.076656+0000","last_became_active":"2024-04-23T01:56:03.539589+0000","last_became_peered":"2024-04-23T01:56:03.539589+0000","last_unstale":"2024-04-23T01:56:50.076656+0000","last_undegraded":"2024-04-23T01:56:50.076656+0000","last_fullsized":"2024-04-23T01:56:50.076656+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3538944,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":432,"num_write_kb":3456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"72'519","reported_seq":554,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:50.368906+0000","last_change":"2024-04-23T01:56:03.635647+0000","last_active":"2024-04-23T01:56:50.368906+0000","last_peered":"2024-04-23T01:56:50.368906+0000","last_clean":"2024-04-23T01:56:50.368906+0000","last_became_active":"2024-04-23T01:56:03.627392+0000","last_became_peered":"2024-04-23T01:56:03.627392+0000","last_unstale":"2024-04-23T01:56:50.368906+0000","last_undegraded":"2024-04-23T01:56:50.368906+0000","last_fullsized":"2024-04-23T01:56:50.368906+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259840,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":520,"num_write_kb":4160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":301,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:34.244954+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:56:34.244954+0000","last_peered":"2024-04-23T01:56:34.244954+0000","last_clean":"2024-04-23T01:56:34.244954+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:56:34.244954+0000","last_undegraded":"2024-04-23T01:56:34.244954+0000","last_fullsized":"2024-04-23T01:56:34.244954+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":168,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:35.582167+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:56:35.582167+0000","last_peered":"2024-04-23T01:56:35.582167+0000","last_clean":"2024-04-23T01:56:35.582167+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:56:35.582167+0000","last_undegraded":"2024-04-23T01:56:35.582167+0000","last_fullsized":"2024-04-23T01:56:35.582167+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.b","version":"72'359","reported_seq":394,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:51.529854+0000","last_change":"2024-04-23T01:56:03.687235+0000","last_active":"2024-04-23T01:56:51.529854+0000","last_peered":"2024-04-23T01:56:51.529854+0000","last_clean":"2024-04-23T01:56:51.529854+0000","last_became_active":"2024-04-23T01:56:03.683119+0000","last_became_peered":"2024-04-23T01:56:03.683119+0000","last_unstale":"2024-04-23T01:56:51.529854+0000","last_undegraded":"2024-04-23T01:56:51.529854+0000","last_fullsized":"2024-04-23T01:56:51.529854+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2949120,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":360,"num_write_kb":2880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":162,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:35.619522+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:56:35.619522+0000","last_peered":"2024-04-23T01:56:35.619522+0000","last_clean":"2024-04-23T01:56:35.619522+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:56:35.619522+0000","last_undegraded":"2024-04-23T01:56:35.619522+0000","last_fullsized":"2024-04-23T01:56:35.619522+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"72'343","reported_seq":378,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:51.054830+0000","last_change":"2024-04-23T01:56:03.560440+0000","last_active":"2024-04-23T01:56:51.054830+0000","last_peered":"2024-04-23T01:56:51.054830+0000","last_clean":"2024-04-23T01:56:51.054830+0000","last_became_active":"2024-04-23T01:56:03.552840+0000","last_became_peered":"2024-04-23T01:56:03.552840+0000","last_unstale":"2024-04-23T01:56:51.054830+0000","last_undegraded":"2024-04-23T01:56:51.054830+0000","last_fullsized":"2024-04-23T01:56:51.054830+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":344,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":168,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:35.560551+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:56:35.560551+0000","last_peered":"2024-04-23T01:56:35.560551+0000","last_clean":"2024-04-23T01:56:35.560551+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:56:35.560551+0000","last_undegraded":"2024-04-23T01:56:35.560551+0000","last_fullsized":"2024-04-23T01:56:35.560551+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"72'407","reported_seq":442,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:51.519515+0000","last_change":"2024-04-23T01:56:03.597772+0000","last_active":"2024-04-23T01:56:51.519515+0000","last_peered":"2024-04-23T01:56:51.519515+0000","last_clean":"2024-04-23T01:56:51.519515+0000","last_became_active":"2024-04-23T01:56:03.589730+0000","last_became_peered":"2024-04-23T01:56:03.589730+0000","last_unstale":"2024-04-23T01:56:51.519515+0000","last_undegraded":"2024-04-23T01:56:51.519515+0000","last_fullsized":"2024-04-23T01:56:51.519515+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342336,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":408,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":162,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:35.632182+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:56:35.632182+0000","last_peered":"2024-04-23T01:56:35.632182+0000","last_clean":"2024-04-23T01:56:35.632182+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:56:35.632182+0000","last_undegraded":"2024-04-23T01:56:35.632182+0000","last_fullsized":"2024-04-23T01:56:35.632182+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.f","version":"72'439","reported_seq":474,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:50.122323+0000","last_change":"2024-04-23T01:56:03.654933+0000","last_active":"2024-04-23T01:56:50.122323+0000","last_peered":"2024-04-23T01:56:50.122323+0000","last_clean":"2024-04-23T01:56:50.122323+0000","last_became_active":"2024-04-23T01:56:03.641185+0000","last_became_peered":"2024-04-23T01:56:03.641185+0000","last_unstale":"2024-04-23T01:56:50.122323+0000","last_undegraded":"2024-04-23T01:56:50.122323+0000","last_fullsized":"2024-04-23T01:56:50.122323+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3604480,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":440,"num_write_kb":3520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":168,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:35.544714+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:56:35.544714+0000","last_peered":"2024-04-23T01:56:35.544714+0000","last_clean":"2024-04-23T01:56:35.544714+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:56:35.544714+0000","last_undegraded":"2024-04-23T01:56:35.544714+0000","last_fullsized":"2024-04-23T01:56:35.544714+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.e","version":"72'399","reported_seq":434,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:51.114734+0000","last_change":"2024-04-23T01:56:03.625197+0000","last_active":"2024-04-23T01:56:51.114734+0000","last_peered":"2024-04-23T01:56:51.114734+0000","last_clean":"2024-04-23T01:56:51.114734+0000","last_became_active":"2024-04-23T01:56:03.611498+0000","last_became_peered":"2024-04-23T01:56:03.611498+0000","last_unstale":"2024-04-23T01:56:51.114734+0000","last_undegraded":"2024-04-23T01:56:51.114734+0000","last_fullsized":"2024-04-23T01:56:51.114734+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3276800,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":400,"num_write_kb":3200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":168,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:34.250754+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:56:34.250754+0000","last_peered":"2024-04-23T01:56:34.250754+0000","last_clean":"2024-04-23T01:56:34.250754+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:56:34.250754+0000","last_undegraded":"2024-04-23T01:56:34.250754+0000","last_fullsized":"2024-04-23T01:56:34.250754+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"72'462","reported_seq":497,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:51.717210+0000","last_change":"2024-04-23T01:56:03.612020+0000","last_active":"2024-04-23T01:56:51.717210+0000","last_peered":"2024-04-23T01:56:51.717210+0000","last_clean":"2024-04-23T01:56:51.717210+0000","last_became_active":"2024-04-23T01:56:03.605401+0000","last_became_peered":"2024-04-23T01:56:03.605401+0000","last_unstale":"2024-04-23T01:56:51.717210+0000","last_undegraded":"2024-04-23T01:56:51.717210+0000","last_fullsized":"2024-04-23T01:56:51.717210+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3792896,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":463,"num_write_kb":3704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":168,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:35.552765+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:56:35.552765+0000","last_peered":"2024-04-23T01:56:35.552765+0000","last_clean":"2024-04-23T01:56:35.552765+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:56:35.552765+0000","last_undegraded":"2024-04-23T01:56:35.552765+0000","last_fullsized":"2024-04-23T01:56:35.552765+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.c","version":"72'359","reported_seq":392,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:51.030016+0000","last_change":"2024-04-23T01:56:03.574136+0000","last_active":"2024-04-23T01:56:51.030016+0000","last_peered":"2024-04-23T01:56:51.030016+0000","last_clean":"2024-04-23T01:56:51.030016+0000","last_became_active":"2024-04-23T01:56:03.566135+0000","last_became_peered":"2024-04-23T01:56:03.566135+0000","last_unstale":"2024-04-23T01:56:51.030016+0000","last_undegraded":"2024-04-23T01:56:51.030016+0000","last_fullsized":"2024-04-23T01:56:51.030016+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2949120,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":360,"num_write_kb":2880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":168,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:35.569657+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:56:35.569657+0000","last_peered":"2024-04-23T01:56:35.569657+0000","last_clean":"2024-04-23T01:56:35.569657+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:56:35.569657+0000","last_undegraded":"2024-04-23T01:56:35.569657+0000","last_fullsized":"2024-04-23T01:56:35.569657+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":54673408,"num_objects":835,"num_object_clones":0,"num_object_copies":2505,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6674,"num_write_kb":53392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":82010112,"data_stored":164020224,"data_compressed":8465976,"data_compressed_allocated":82010112,"data_compressed_original":164020224,"omap_allocated":0,"internal_metadata":0},"log_size":6658,"ondisk_log_size":6658,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706368,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":125664,"kb_used_data":22484,"kb_used_omap":0,"kb_used_meta":103168,"kb_avail":94246176,"statfs":{"total":96636764160,"available":96508084224,"internally_reserved":0,"allocated":23023616,"data_stored":45503490,"data_compressed":2336500,"data_compressed_allocated":22384640,"data_compressed_original":45260800,"omap_allocated":0,"internal_metadata":105644032},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706368,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":120292,"kb_used_data":17116,"kb_used_omap":0,"kb_used_meta":103168,"kb_avail":94251548,"statfs":{"total":96636764160,"available":96513585152,"internally_reserved":0,"allocated":17526784,"data_stored":34509826,"data_compressed":1769052,"data_compressed_allocated":16887808,"data_compressed_original":34267136,"omap_allocated":0,"internal_metadata":105644032},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706368,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":124744,"kb_used_data":17408,"kb_used_omap":0,"kb_used_meta":107328,"kb_avail":94247096,"statfs":{"total":96636764160,"available":96509026304,"internally_reserved":0,"allocated":17825792,"data_stored":34640354,"data_compressed":1776325,"data_compressed_allocated":17207296,"data_compressed_original":34414592,"omap_allocated":0,"internal_metadata":109903872},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706367,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":132300,"kb_used_data":24964,"kb_used_omap":0,"kb_used_meta":107328,"kb_avail":94239540,"statfs":{"total":96636764160,"available":96501288960,"internally_reserved":0,"allocated":25563136,"data_stored":50582530,"data_compressed":2598664,"data_compressed_allocated":24924160,"data_compressed_original":50339840,"omap_allocated":0,"internal_metadata":109903872},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":25415680,"data_stored":50831360,"data_compressed":2623701,"data_compressed_allocated":25415680,"data_compressed_original":50831360,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":17215488,"data_stored":34430976,"data_compressed":1777171,"data_compressed_allocated":17215488,"data_compressed_original":34430976,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16945152,"data_stored":33890304,"data_compressed":1749251,"data_compressed_allocated":16945152,"data_compressed_original":33890304,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":22433792,"data_stored":44867584,"data_compressed":2315853,"data_compressed_allocated":22433792,"data_compressed_original":44867584,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:56:54.976 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:56:55.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:55.104+0000 7ff02a06d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:55.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:55.110+0000 7ff02a06d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:55.479 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 51 16 7194 7178 1.09947 1.15625 0.0663685 0.113493 2024-04-23T01:56:56.479 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 52 16 7329 7313 1.09861 1.05469 0.159796 0.113659 2024-04-23T01:56:57.479 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 53 16 7459 7443 1.09704 1.01562 0.180762 0.113773 2024-04-23T01:56:57.487 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:56:57.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:57.605+0000 7f081186d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:57.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:57.611+0000 7f081186d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:58.480 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 54 16 7593 7577 1.09611 1.04688 0.10085 0.113924 2024-04-23T01:56:59.480 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 55 16 7720 7704 1.09422 0.992188 0.0319067 0.114067 2024-04-23T01:56:59.566 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:56:59.567 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-23T01:56:59.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:59.692+0000 7fa5a53ff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:59.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:59.706+0000 7fa59ffff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:59.782 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:56:59.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:59.917+0000 7f0fa4a64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:56:59.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:56:59.927+0000 7f0fa4a64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:00.287 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.0 to 1 (10000) 2024-04-23T01:57:00.305 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-23T01:57:00.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:00.429+0000 7f3593aed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:00.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:00.441+0000 7f3593aed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:00.480 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 56 16 7848 7832 1.09254 1 0.049091 0.114274 2024-04-23T01:57:01.291 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.1 to 1 (10000) 2024-04-23T01:57:01.309 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-23T01:57:01.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:01.431+0000 7efe9555d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:01.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:01.443+0000 7efe9555d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:01.480 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 57 16 7980 7964 1.09146 1.03125 0.0583887 0.114361 2024-04-23T01:57:02.241 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:57:02.295 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.2 to 1 (10000) 2024-04-23T01:57:02.316 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-23T01:57:02.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:02.363+0000 7f4d31c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:02.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:02.368+0000 7f4d31c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:02.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:02.444+0000 7fe09062d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:02.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:02.459+0000 7fe09062d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:02.480 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 58 16 8119 8103 1.09136 1.08594 0.201618 0.114334 2024-04-23T01:57:03.302 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.3 to 1 (10000) 2024-04-23T01:57:03.320 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-23T01:57:03.321 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:57:03.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:03.454+0000 7efd1b62d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:03.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:03.465+0000 7efd1b62d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:03.480 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 59 16 8245 8229 1.08955 0.984375 0.202677 0.114543 2024-04-23T01:57:03.742 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:57:03.742 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:57:03.763 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":428,"stamp":"2024-04-23T01:57:03.687894+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":66183712,"num_objects":1005,"num_object_clones":0,"num_object_copies":3015,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":8144,"num_write_kb":65950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8091,"ondisk_log_size":8091,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":535588,"kb_used_data":97920,"kb_used_omap":0,"kb_used_meta":437632,"kb_avail":376951772,"statfs":{"total":386547056640,"available":385998614528,"internally_reserved":0,"allocated":100270080,"data_stored":197845148,"data_compressed":10163658,"data_compressed_allocated":97701888,"data_compressed_original":196878336,"omap_allocated":0,"internal_metadata":448135168},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5414912,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":661,"num_write_kb":5288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000789"},"pg_stats":[{"pgid":"9.7","version":"72'632","reported_seq":669,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:00.331265+0000","last_change":"2024-04-23T01:56:03.683930+0000","last_active":"2024-04-23T01:57:00.331265+0000","last_peered":"2024-04-23T01:57:00.331265+0000","last_clean":"2024-04-23T01:57:00.331265+0000","last_became_active":"2024-04-23T01:56:03.675554+0000","last_became_peered":"2024-04-23T01:56:03.675554+0000","last_unstale":"2024-04-23T01:57:00.331265+0000","last_undegraded":"2024-04-23T01:57:00.331265+0000","last_fullsized":"2024-04-23T01:57:00.331265+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":5056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"74'585","reported_seq":622,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.668597+0000","last_change":"2024-04-23T01:56:03.552647+0000","last_active":"2024-04-23T01:57:01.668597+0000","last_peered":"2024-04-23T01:57:01.668597+0000","last_clean":"2024-04-23T01:57:01.668597+0000","last_became_active":"2024-04-23T01:56:03.541685+0000","last_became_peered":"2024-04-23T01:56:03.541685+0000","last_unstale":"2024-04-23T01:57:01.668597+0000","last_undegraded":"2024-04-23T01:57:01.668597+0000","last_fullsized":"2024-04-23T01:57:01.668597+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":585,"log_dups_size":0,"ondisk_log_size":585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4800512,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":586,"num_write_kb":4688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"72'503","reported_seq":538,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:59.401467+0000","last_change":"2024-04-23T01:56:03.515568+0000","last_active":"2024-04-23T01:56:59.401467+0000","last_peered":"2024-04-23T01:56:59.401467+0000","last_clean":"2024-04-23T01:56:59.401467+0000","last_became_active":"2024-04-23T01:56:03.510354+0000","last_became_peered":"2024-04-23T01:56:03.510354+0000","last_unstale":"2024-04-23T01:56:59.401467+0000","last_undegraded":"2024-04-23T01:56:59.401467+0000","last_fullsized":"2024-04-23T01:56:59.401467+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4128768,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":504,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.4","version":"72'495","reported_seq":530,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:57:00.066489+0000","last_change":"2024-04-23T01:56:03.647359+0000","last_active":"2024-04-23T01:57:00.066489+0000","last_peered":"2024-04-23T01:57:00.066489+0000","last_clean":"2024-04-23T01:57:00.066489+0000","last_became_active":"2024-04-23T01:56:03.642592+0000","last_became_peered":"2024-04-23T01:56:03.642592+0000","last_unstale":"2024-04-23T01:57:00.066489+0000","last_undegraded":"2024-04-23T01:57:00.066489+0000","last_fullsized":"2024-04-23T01:57:00.066489+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4063232,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":496,"num_write_kb":3968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"73'487","reported_seq":525,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.661737+0000","last_change":"2024-04-23T01:56:03.566150+0000","last_active":"2024-04-23T01:57:01.661737+0000","last_peered":"2024-04-23T01:57:01.661737+0000","last_clean":"2024-04-23T01:57:01.661737+0000","last_became_active":"2024-04-23T01:56:03.553086+0000","last_became_peered":"2024-04-23T01:56:03.553086+0000","last_unstale":"2024-04-23T01:57:01.661737+0000","last_undegraded":"2024-04-23T01:57:01.661737+0000","last_fullsized":"2024-04-23T01:57:01.661737+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3997696,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":488,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"74'388","reported_seq":425,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.652273+0000","last_change":"2024-04-23T01:56:03.605892+0000","last_active":"2024-04-23T01:57:01.652273+0000","last_peered":"2024-04-23T01:57:01.652273+0000","last_clean":"2024-04-23T01:57:01.652273+0000","last_became_active":"2024-04-23T01:56:03.601950+0000","last_became_peered":"2024-04-23T01:56:03.601950+0000","last_unstale":"2024-04-23T01:57:01.652273+0000","last_undegraded":"2024-04-23T01:57:01.652273+0000","last_fullsized":"2024-04-23T01:57:01.652273+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3186688,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":389,"num_write_kb":3112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"73'335","reported_seq":373,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.149652+0000","last_change":"2024-04-23T01:56:03.618761+0000","last_active":"2024-04-23T01:57:01.149652+0000","last_peered":"2024-04-23T01:57:01.149652+0000","last_clean":"2024-04-23T01:57:01.149652+0000","last_became_active":"2024-04-23T01:56:03.602521+0000","last_became_peered":"2024-04-23T01:56:03.602521+0000","last_unstale":"2024-04-23T01:57:01.149652+0000","last_undegraded":"2024-04-23T01:57:01.149652+0000","last_fullsized":"2024-04-23T01:57:01.149652+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"72'517","reported_seq":552,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.738746+0000","last_change":"2024-04-23T01:56:03.544149+0000","last_active":"2024-04-23T01:57:01.738746+0000","last_peered":"2024-04-23T01:57:01.738746+0000","last_clean":"2024-04-23T01:57:01.738746+0000","last_became_active":"2024-04-23T01:56:03.539589+0000","last_became_peered":"2024-04-23T01:56:03.539589+0000","last_unstale":"2024-04-23T01:57:01.738746+0000","last_undegraded":"2024-04-23T01:57:01.738746+0000","last_fullsized":"2024-04-23T01:57:01.738746+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4243456,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":518,"num_write_kb":4144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"73'625","reported_seq":663,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.714619+0000","last_change":"2024-04-23T01:56:03.635647+0000","last_active":"2024-04-23T01:57:01.714619+0000","last_peered":"2024-04-23T01:57:01.714619+0000","last_clean":"2024-04-23T01:57:01.714619+0000","last_became_active":"2024-04-23T01:56:03.627392+0000","last_became_peered":"2024-04-23T01:56:03.627392+0000","last_unstale":"2024-04-23T01:57:01.714619+0000","last_undegraded":"2024-04-23T01:57:01.714619+0000","last_fullsized":"2024-04-23T01:57:01.714619+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5128192,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":626,"num_write_kb":5008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":304,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:00.858838+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:57:00.858838+0000","last_peered":"2024-04-23T01:57:00.858838+0000","last_clean":"2024-04-23T01:57:00.858838+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:57:00.858838+0000","last_undegraded":"2024-04-23T01:57:00.858838+0000","last_fullsized":"2024-04-23T01:57:00.858838+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":172,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.357059+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:57:01.357059+0000","last_peered":"2024-04-23T01:57:01.357059+0000","last_clean":"2024-04-23T01:57:01.357059+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:57:01.357059+0000","last_undegraded":"2024-04-23T01:57:01.357059+0000","last_fullsized":"2024-04-23T01:57:01.357059+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.b","version":"72'456","reported_seq":493,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:00.349583+0000","last_change":"2024-04-23T01:56:03.687235+0000","last_active":"2024-04-23T01:57:00.349583+0000","last_peered":"2024-04-23T01:57:00.349583+0000","last_clean":"2024-04-23T01:57:00.349583+0000","last_became_active":"2024-04-23T01:56:03.683119+0000","last_became_peered":"2024-04-23T01:56:03.683119+0000","last_unstale":"2024-04-23T01:57:00.349583+0000","last_undegraded":"2024-04-23T01:57:00.349583+0000","last_fullsized":"2024-04-23T01:57:00.349583+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":456,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":165,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:00.348406+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:57:00.348406+0000","last_peered":"2024-04-23T01:57:00.348406+0000","last_clean":"2024-04-23T01:57:00.348406+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:57:00.348406+0000","last_undegraded":"2024-04-23T01:57:00.348406+0000","last_fullsized":"2024-04-23T01:57:00.348406+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"72'447","reported_seq":482,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:57:00.939516+0000","last_change":"2024-04-23T01:56:03.560440+0000","last_active":"2024-04-23T01:57:00.939516+0000","last_peered":"2024-04-23T01:57:00.939516+0000","last_clean":"2024-04-23T01:57:00.939516+0000","last_became_active":"2024-04-23T01:56:03.552840+0000","last_became_peered":"2024-04-23T01:56:03.552840+0000","last_unstale":"2024-04-23T01:57:00.939516+0000","last_undegraded":"2024-04-23T01:57:00.939516+0000","last_fullsized":"2024-04-23T01:57:00.939516+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":448,"num_write_kb":3584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":172,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.328035+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:57:01.328035+0000","last_peered":"2024-04-23T01:57:01.328035+0000","last_clean":"2024-04-23T01:57:01.328035+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:57:01.328035+0000","last_undegraded":"2024-04-23T01:57:01.328035+0000","last_fullsized":"2024-04-23T01:57:01.328035+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"72'488","reported_seq":525,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:00.854548+0000","last_change":"2024-04-23T01:56:03.597772+0000","last_active":"2024-04-23T01:57:00.854548+0000","last_peered":"2024-04-23T01:57:00.854548+0000","last_clean":"2024-04-23T01:57:00.854548+0000","last_became_active":"2024-04-23T01:56:03.589730+0000","last_became_peered":"2024-04-23T01:56:03.589730+0000","last_unstale":"2024-04-23T01:57:00.854548+0000","last_undegraded":"2024-04-23T01:57:00.854548+0000","last_fullsized":"2024-04-23T01:57:00.854548+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3997696,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":488,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":165,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:00.321885+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:57:00.321885+0000","last_peered":"2024-04-23T01:57:00.321885+0000","last_clean":"2024-04-23T01:57:00.321885+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:57:00.321885+0000","last_undegraded":"2024-04-23T01:57:00.321885+0000","last_fullsized":"2024-04-23T01:57:00.321885+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.f","version":"73'479","reported_seq":517,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:00.386073+0000","last_change":"2024-04-23T01:56:03.654933+0000","last_active":"2024-04-23T01:57:00.386073+0000","last_peered":"2024-04-23T01:57:00.386073+0000","last_clean":"2024-04-23T01:57:00.386073+0000","last_became_active":"2024-04-23T01:56:03.641185+0000","last_became_peered":"2024-04-23T01:56:03.641185+0000","last_unstale":"2024-04-23T01:57:00.386073+0000","last_undegraded":"2024-04-23T01:57:00.386073+0000","last_fullsized":"2024-04-23T01:57:00.386073+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":480,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":168,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:35.544714+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:56:35.544714+0000","last_peered":"2024-04-23T01:56:35.544714+0000","last_clean":"2024-04-23T01:56:35.544714+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:56:35.544714+0000","last_undegraded":"2024-04-23T01:56:35.544714+0000","last_fullsized":"2024-04-23T01:56:35.544714+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.e","version":"73'511","reported_seq":549,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.033071+0000","last_change":"2024-04-23T01:56:03.625197+0000","last_active":"2024-04-23T01:57:01.033071+0000","last_peered":"2024-04-23T01:57:01.033071+0000","last_clean":"2024-04-23T01:57:01.033071+0000","last_became_active":"2024-04-23T01:56:03.611498+0000","last_became_peered":"2024-04-23T01:56:03.611498+0000","last_unstale":"2024-04-23T01:57:01.033071+0000","last_undegraded":"2024-04-23T01:57:01.033071+0000","last_fullsized":"2024-04-23T01:57:01.033071+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":512,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":171,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:00.867573+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:57:00.867573+0000","last_peered":"2024-04-23T01:57:00.867573+0000","last_clean":"2024-04-23T01:57:00.867573+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:57:00.867573+0000","last_undegraded":"2024-04-23T01:57:00.867573+0000","last_fullsized":"2024-04-23T01:57:00.867573+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"73'591","reported_seq":629,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.537606+0000","last_change":"2024-04-23T01:56:03.612020+0000","last_active":"2024-04-23T01:57:01.537606+0000","last_peered":"2024-04-23T01:57:01.537606+0000","last_clean":"2024-04-23T01:57:01.537606+0000","last_became_active":"2024-04-23T01:56:03.605401+0000","last_became_peered":"2024-04-23T01:56:03.605401+0000","last_unstale":"2024-04-23T01:57:01.537606+0000","last_undegraded":"2024-04-23T01:57:01.537606+0000","last_fullsized":"2024-04-23T01:57:01.537606+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":592,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":168,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:35.552765+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:56:35.552765+0000","last_peered":"2024-04-23T01:56:35.552765+0000","last_clean":"2024-04-23T01:56:35.552765+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:56:35.552765+0000","last_undegraded":"2024-04-23T01:56:35.552765+0000","last_fullsized":"2024-04-23T01:56:35.552765+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.c","version":"74'455","reported_seq":492,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.625172+0000","last_change":"2024-04-23T01:56:03.574136+0000","last_active":"2024-04-23T01:57:01.625172+0000","last_peered":"2024-04-23T01:57:01.625172+0000","last_clean":"2024-04-23T01:57:01.625172+0000","last_became_active":"2024-04-23T01:56:03.566135+0000","last_became_peered":"2024-04-23T01:56:03.566135+0000","last_unstale":"2024-04-23T01:57:01.625172+0000","last_undegraded":"2024-04-23T01:57:01.625172+0000","last_fullsized":"2024-04-23T01:57:01.625172+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":456,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":172,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.354076+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:57:01.354076+0000","last_peered":"2024-04-23T01:57:01.354076+0000","last_clean":"2024-04-23T01:57:01.354076+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:57:01.354076+0000","last_undegraded":"2024-04-23T01:57:01.354076+0000","last_fullsized":"2024-04-23T01:57:01.354076+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":65593344,"num_objects":1003,"num_object_clones":0,"num_object_copies":3009,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8007,"num_write_kb":64056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":98365440,"data_stored":196730880,"data_compressed":10155015,"data_compressed_allocated":98365440,"data_compressed_original":196730880,"omap_allocated":0,"internal_metadata":0},"log_size":7994,"ondisk_log_size":7994,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706378,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":134300,"kb_used_data":26964,"kb_used_omap":0,"kb_used_meta":107328,"kb_avail":94237540,"statfs":{"total":96636764160,"available":96499240960,"internally_reserved":0,"allocated":27611136,"data_stored":54665391,"data_compressed":2809414,"data_compressed_allocated":26963968,"data_compressed_original":54419456,"omap_allocated":0,"internal_metadata":109903872},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706378,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":127932,"kb_used_data":20592,"kb_used_omap":0,"kb_used_meta":107328,"kb_avail":94243908,"statfs":{"total":96636764160,"available":96505761792,"internally_reserved":0,"allocated":21086208,"data_stored":41628674,"data_compressed":2136639,"data_compressed_allocated":20447232,"data_compressed_original":41385984,"omap_allocated":0,"internal_metadata":109903872},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706378,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":132020,"kb_used_data":20524,"kb_used_omap":0,"kb_used_meta":111488,"kb_avail":94239820,"statfs":{"total":96636764160,"available":96501575680,"internally_reserved":0,"allocated":21016576,"data_stored":40995644,"data_compressed":2104150,"data_compressed_allocated":20381696,"data_compressed_original":40763392,"omap_allocated":0,"internal_metadata":114163712},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706377,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":141336,"kb_used_data":29840,"kb_used_omap":0,"kb_used_meta":111488,"kb_avail":94230504,"statfs":{"total":96636764160,"available":96492036096,"internally_reserved":0,"allocated":30556160,"data_stored":60555439,"data_compressed":3113455,"data_compressed_allocated":29908992,"data_compressed_original":60309504,"omap_allocated":0,"internal_metadata":114163712},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":30363648,"data_stored":60727296,"data_compressed":3134685,"data_compressed_allocated":30363648,"data_compressed_original":60727296,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20422656,"data_stored":40845312,"data_compressed":2108380,"data_compressed_allocated":20422656,"data_compressed_original":40845312,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20586496,"data_stored":41172992,"data_compressed":2125298,"data_compressed_allocated":20586496,"data_compressed_original":41172992,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":26992640,"data_stored":53985280,"data_compressed":2786652,"data_compressed_allocated":26992640,"data_compressed_original":53985280,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:57:03.764 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:57:03.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:03.884+0000 7fb7e406d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:03.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:03.894+0000 7fb7e406d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:04.112 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:57:04.113 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:57:04.129 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":428,"stamp":"2024-04-23T01:57:03.687894+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":66183712,"num_objects":1005,"num_object_clones":0,"num_object_copies":3015,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":8144,"num_write_kb":65950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8091,"ondisk_log_size":8091,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":535588,"kb_used_data":97920,"kb_used_omap":0,"kb_used_meta":437632,"kb_avail":376951772,"statfs":{"total":386547056640,"available":385998614528,"internally_reserved":0,"allocated":100270080,"data_stored":197845148,"data_compressed":10163658,"data_compressed_allocated":97701888,"data_compressed_original":196878336,"omap_allocated":0,"internal_metadata":448135168},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5414912,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":661,"num_write_kb":5288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000789"},"pg_stats":[{"pgid":"9.7","version":"72'632","reported_seq":669,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:00.331265+0000","last_change":"2024-04-23T01:56:03.683930+0000","last_active":"2024-04-23T01:57:00.331265+0000","last_peered":"2024-04-23T01:57:00.331265+0000","last_clean":"2024-04-23T01:57:00.331265+0000","last_became_active":"2024-04-23T01:56:03.675554+0000","last_became_peered":"2024-04-23T01:56:03.675554+0000","last_unstale":"2024-04-23T01:57:00.331265+0000","last_undegraded":"2024-04-23T01:57:00.331265+0000","last_fullsized":"2024-04-23T01:57:00.331265+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":5056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"74'585","reported_seq":622,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.668597+0000","last_change":"2024-04-23T01:56:03.552647+0000","last_active":"2024-04-23T01:57:01.668597+0000","last_peered":"2024-04-23T01:57:01.668597+0000","last_clean":"2024-04-23T01:57:01.668597+0000","last_became_active":"2024-04-23T01:56:03.541685+0000","last_became_peered":"2024-04-23T01:56:03.541685+0000","last_unstale":"2024-04-23T01:57:01.668597+0000","last_undegraded":"2024-04-23T01:57:01.668597+0000","last_fullsized":"2024-04-23T01:57:01.668597+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":585,"log_dups_size":0,"ondisk_log_size":585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4800512,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":586,"num_write_kb":4688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"72'503","reported_seq":538,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:59.401467+0000","last_change":"2024-04-23T01:56:03.515568+0000","last_active":"2024-04-23T01:56:59.401467+0000","last_peered":"2024-04-23T01:56:59.401467+0000","last_clean":"2024-04-23T01:56:59.401467+0000","last_became_active":"2024-04-23T01:56:03.510354+0000","last_became_peered":"2024-04-23T01:56:03.510354+0000","last_unstale":"2024-04-23T01:56:59.401467+0000","last_undegraded":"2024-04-23T01:56:59.401467+0000","last_fullsized":"2024-04-23T01:56:59.401467+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4128768,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":504,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.4","version":"72'495","reported_seq":530,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:57:00.066489+0000","last_change":"2024-04-23T01:56:03.647359+0000","last_active":"2024-04-23T01:57:00.066489+0000","last_peered":"2024-04-23T01:57:00.066489+0000","last_clean":"2024-04-23T01:57:00.066489+0000","last_became_active":"2024-04-23T01:56:03.642592+0000","last_became_peered":"2024-04-23T01:56:03.642592+0000","last_unstale":"2024-04-23T01:57:00.066489+0000","last_undegraded":"2024-04-23T01:57:00.066489+0000","last_fullsized":"2024-04-23T01:57:00.066489+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4063232,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":496,"num_write_kb":3968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"73'487","reported_seq":525,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.661737+0000","last_change":"2024-04-23T01:56:03.566150+0000","last_active":"2024-04-23T01:57:01.661737+0000","last_peered":"2024-04-23T01:57:01.661737+0000","last_clean":"2024-04-23T01:57:01.661737+0000","last_became_active":"2024-04-23T01:56:03.553086+0000","last_became_peered":"2024-04-23T01:56:03.553086+0000","last_unstale":"2024-04-23T01:57:01.661737+0000","last_undegraded":"2024-04-23T01:57:01.661737+0000","last_fullsized":"2024-04-23T01:57:01.661737+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3997696,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":488,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"74'388","reported_seq":425,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.652273+0000","last_change":"2024-04-23T01:56:03.605892+0000","last_active":"2024-04-23T01:57:01.652273+0000","last_peered":"2024-04-23T01:57:01.652273+0000","last_clean":"2024-04-23T01:57:01.652273+0000","last_became_active":"2024-04-23T01:56:03.601950+0000","last_became_peered":"2024-04-23T01:56:03.601950+0000","last_unstale":"2024-04-23T01:57:01.652273+0000","last_undegraded":"2024-04-23T01:57:01.652273+0000","last_fullsized":"2024-04-23T01:57:01.652273+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3186688,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":389,"num_write_kb":3112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"73'335","reported_seq":373,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.149652+0000","last_change":"2024-04-23T01:56:03.618761+0000","last_active":"2024-04-23T01:57:01.149652+0000","last_peered":"2024-04-23T01:57:01.149652+0000","last_clean":"2024-04-23T01:57:01.149652+0000","last_became_active":"2024-04-23T01:56:03.602521+0000","last_became_peered":"2024-04-23T01:56:03.602521+0000","last_unstale":"2024-04-23T01:57:01.149652+0000","last_undegraded":"2024-04-23T01:57:01.149652+0000","last_fullsized":"2024-04-23T01:57:01.149652+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"72'517","reported_seq":552,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.738746+0000","last_change":"2024-04-23T01:56:03.544149+0000","last_active":"2024-04-23T01:57:01.738746+0000","last_peered":"2024-04-23T01:57:01.738746+0000","last_clean":"2024-04-23T01:57:01.738746+0000","last_became_active":"2024-04-23T01:56:03.539589+0000","last_became_peered":"2024-04-23T01:56:03.539589+0000","last_unstale":"2024-04-23T01:57:01.738746+0000","last_undegraded":"2024-04-23T01:57:01.738746+0000","last_fullsized":"2024-04-23T01:57:01.738746+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4243456,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":518,"num_write_kb":4144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"73'625","reported_seq":663,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.714619+0000","last_change":"2024-04-23T01:56:03.635647+0000","last_active":"2024-04-23T01:57:01.714619+0000","last_peered":"2024-04-23T01:57:01.714619+0000","last_clean":"2024-04-23T01:57:01.714619+0000","last_became_active":"2024-04-23T01:56:03.627392+0000","last_became_peered":"2024-04-23T01:56:03.627392+0000","last_unstale":"2024-04-23T01:57:01.714619+0000","last_undegraded":"2024-04-23T01:57:01.714619+0000","last_fullsized":"2024-04-23T01:57:01.714619+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5128192,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":626,"num_write_kb":5008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":304,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:00.858838+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:57:00.858838+0000","last_peered":"2024-04-23T01:57:00.858838+0000","last_clean":"2024-04-23T01:57:00.858838+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:57:00.858838+0000","last_undegraded":"2024-04-23T01:57:00.858838+0000","last_fullsized":"2024-04-23T01:57:00.858838+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":172,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.357059+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:57:01.357059+0000","last_peered":"2024-04-23T01:57:01.357059+0000","last_clean":"2024-04-23T01:57:01.357059+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:57:01.357059+0000","last_undegraded":"2024-04-23T01:57:01.357059+0000","last_fullsized":"2024-04-23T01:57:01.357059+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.b","version":"72'456","reported_seq":493,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:00.349583+0000","last_change":"2024-04-23T01:56:03.687235+0000","last_active":"2024-04-23T01:57:00.349583+0000","last_peered":"2024-04-23T01:57:00.349583+0000","last_clean":"2024-04-23T01:57:00.349583+0000","last_became_active":"2024-04-23T01:56:03.683119+0000","last_became_peered":"2024-04-23T01:56:03.683119+0000","last_unstale":"2024-04-23T01:57:00.349583+0000","last_undegraded":"2024-04-23T01:57:00.349583+0000","last_fullsized":"2024-04-23T01:57:00.349583+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":456,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":165,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:00.348406+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:57:00.348406+0000","last_peered":"2024-04-23T01:57:00.348406+0000","last_clean":"2024-04-23T01:57:00.348406+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:57:00.348406+0000","last_undegraded":"2024-04-23T01:57:00.348406+0000","last_fullsized":"2024-04-23T01:57:00.348406+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"72'447","reported_seq":482,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:57:00.939516+0000","last_change":"2024-04-23T01:56:03.560440+0000","last_active":"2024-04-23T01:57:00.939516+0000","last_peered":"2024-04-23T01:57:00.939516+0000","last_clean":"2024-04-23T01:57:00.939516+0000","last_became_active":"2024-04-23T01:56:03.552840+0000","last_became_peered":"2024-04-23T01:56:03.552840+0000","last_unstale":"2024-04-23T01:57:00.939516+0000","last_undegraded":"2024-04-23T01:57:00.939516+0000","last_fullsized":"2024-04-23T01:57:00.939516+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":448,"num_write_kb":3584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":172,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.328035+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:57:01.328035+0000","last_peered":"2024-04-23T01:57:01.328035+0000","last_clean":"2024-04-23T01:57:01.328035+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:57:01.328035+0000","last_undegraded":"2024-04-23T01:57:01.328035+0000","last_fullsized":"2024-04-23T01:57:01.328035+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"72'488","reported_seq":525,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:00.854548+0000","last_change":"2024-04-23T01:56:03.597772+0000","last_active":"2024-04-23T01:57:00.854548+0000","last_peered":"2024-04-23T01:57:00.854548+0000","last_clean":"2024-04-23T01:57:00.854548+0000","last_became_active":"2024-04-23T01:56:03.589730+0000","last_became_peered":"2024-04-23T01:56:03.589730+0000","last_unstale":"2024-04-23T01:57:00.854548+0000","last_undegraded":"2024-04-23T01:57:00.854548+0000","last_fullsized":"2024-04-23T01:57:00.854548+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3997696,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":488,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":165,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:00.321885+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:57:00.321885+0000","last_peered":"2024-04-23T01:57:00.321885+0000","last_clean":"2024-04-23T01:57:00.321885+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:57:00.321885+0000","last_undegraded":"2024-04-23T01:57:00.321885+0000","last_fullsized":"2024-04-23T01:57:00.321885+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.f","version":"73'479","reported_seq":517,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:00.386073+0000","last_change":"2024-04-23T01:56:03.654933+0000","last_active":"2024-04-23T01:57:00.386073+0000","last_peered":"2024-04-23T01:57:00.386073+0000","last_clean":"2024-04-23T01:57:00.386073+0000","last_became_active":"2024-04-23T01:56:03.641185+0000","last_became_peered":"2024-04-23T01:56:03.641185+0000","last_unstale":"2024-04-23T01:57:00.386073+0000","last_undegraded":"2024-04-23T01:57:00.386073+0000","last_fullsized":"2024-04-23T01:57:00.386073+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":480,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":168,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:35.544714+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:56:35.544714+0000","last_peered":"2024-04-23T01:56:35.544714+0000","last_clean":"2024-04-23T01:56:35.544714+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:56:35.544714+0000","last_undegraded":"2024-04-23T01:56:35.544714+0000","last_fullsized":"2024-04-23T01:56:35.544714+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.e","version":"73'511","reported_seq":549,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.033071+0000","last_change":"2024-04-23T01:56:03.625197+0000","last_active":"2024-04-23T01:57:01.033071+0000","last_peered":"2024-04-23T01:57:01.033071+0000","last_clean":"2024-04-23T01:57:01.033071+0000","last_became_active":"2024-04-23T01:56:03.611498+0000","last_became_peered":"2024-04-23T01:56:03.611498+0000","last_unstale":"2024-04-23T01:57:01.033071+0000","last_undegraded":"2024-04-23T01:57:01.033071+0000","last_fullsized":"2024-04-23T01:57:01.033071+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":512,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":171,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:00.867573+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:57:00.867573+0000","last_peered":"2024-04-23T01:57:00.867573+0000","last_clean":"2024-04-23T01:57:00.867573+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:57:00.867573+0000","last_undegraded":"2024-04-23T01:57:00.867573+0000","last_fullsized":"2024-04-23T01:57:00.867573+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"73'591","reported_seq":629,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.537606+0000","last_change":"2024-04-23T01:56:03.612020+0000","last_active":"2024-04-23T01:57:01.537606+0000","last_peered":"2024-04-23T01:57:01.537606+0000","last_clean":"2024-04-23T01:57:01.537606+0000","last_became_active":"2024-04-23T01:56:03.605401+0000","last_became_peered":"2024-04-23T01:56:03.605401+0000","last_unstale":"2024-04-23T01:57:01.537606+0000","last_undegraded":"2024-04-23T01:57:01.537606+0000","last_fullsized":"2024-04-23T01:57:01.537606+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":592,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":168,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:35.552765+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:56:35.552765+0000","last_peered":"2024-04-23T01:56:35.552765+0000","last_clean":"2024-04-23T01:56:35.552765+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:56:35.552765+0000","last_undegraded":"2024-04-23T01:56:35.552765+0000","last_fullsized":"2024-04-23T01:56:35.552765+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.c","version":"74'455","reported_seq":492,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.625172+0000","last_change":"2024-04-23T01:56:03.574136+0000","last_active":"2024-04-23T01:57:01.625172+0000","last_peered":"2024-04-23T01:57:01.625172+0000","last_clean":"2024-04-23T01:57:01.625172+0000","last_became_active":"2024-04-23T01:56:03.566135+0000","last_became_peered":"2024-04-23T01:56:03.566135+0000","last_unstale":"2024-04-23T01:57:01.625172+0000","last_undegraded":"2024-04-23T01:57:01.625172+0000","last_fullsized":"2024-04-23T01:57:01.625172+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":456,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":172,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.354076+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:57:01.354076+0000","last_peered":"2024-04-23T01:57:01.354076+0000","last_clean":"2024-04-23T01:57:01.354076+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:57:01.354076+0000","last_undegraded":"2024-04-23T01:57:01.354076+0000","last_fullsized":"2024-04-23T01:57:01.354076+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":65593344,"num_objects":1003,"num_object_clones":0,"num_object_copies":3009,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8007,"num_write_kb":64056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":98365440,"data_stored":196730880,"data_compressed":10155015,"data_compressed_allocated":98365440,"data_compressed_original":196730880,"omap_allocated":0,"internal_metadata":0},"log_size":7994,"ondisk_log_size":7994,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706378,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":134300,"kb_used_data":26964,"kb_used_omap":0,"kb_used_meta":107328,"kb_avail":94237540,"statfs":{"total":96636764160,"available":96499240960,"internally_reserved":0,"allocated":27611136,"data_stored":54665391,"data_compressed":2809414,"data_compressed_allocated":26963968,"data_compressed_original":54419456,"omap_allocated":0,"internal_metadata":109903872},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706378,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":127932,"kb_used_data":20592,"kb_used_omap":0,"kb_used_meta":107328,"kb_avail":94243908,"statfs":{"total":96636764160,"available":96505761792,"internally_reserved":0,"allocated":21086208,"data_stored":41628674,"data_compressed":2136639,"data_compressed_allocated":20447232,"data_compressed_original":41385984,"omap_allocated":0,"internal_metadata":109903872},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706378,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":132020,"kb_used_data":20524,"kb_used_omap":0,"kb_used_meta":111488,"kb_avail":94239820,"statfs":{"total":96636764160,"available":96501575680,"internally_reserved":0,"allocated":21016576,"data_stored":40995644,"data_compressed":2104150,"data_compressed_allocated":20381696,"data_compressed_original":40763392,"omap_allocated":0,"internal_metadata":114163712},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706377,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":141336,"kb_used_data":29840,"kb_used_omap":0,"kb_used_meta":111488,"kb_avail":94230504,"statfs":{"total":96636764160,"available":96492036096,"internally_reserved":0,"allocated":30556160,"data_stored":60555439,"data_compressed":3113455,"data_compressed_allocated":29908992,"data_compressed_original":60309504,"omap_allocated":0,"internal_metadata":114163712},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":30363648,"data_stored":60727296,"data_compressed":3134685,"data_compressed_allocated":30363648,"data_compressed_original":60727296,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20422656,"data_stored":40845312,"data_compressed":2108380,"data_compressed_allocated":20422656,"data_compressed_original":40845312,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20586496,"data_stored":41172992,"data_compressed":2125298,"data_compressed_allocated":20586496,"data_compressed_original":41172992,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":26992640,"data_stored":53985280,"data_compressed":2786652,"data_compressed_allocated":26992640,"data_compressed_original":53985280,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:57:04.131 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-23T01:57:04.131 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:57:04.131 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:57:04.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:04.256+0000 7f7ee702d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:04.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:04.266+0000 7f7ee702d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:04.480 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:57:04.479701+0000 min lat: 0.014953 max lat: 0.38169 avg lat: 0.114593 2024-04-23T01:57:04.480 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:57:04.480 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 60 16 8382 8366 1.08923 1.07031 0.133953 0.114593 2024-04-23T01:57:04.517 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:57:04.517 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:57:04.527 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:57:04.535 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":428,"stamp":"2024-04-23T01:57:03.687894+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":66183712,"num_objects":1005,"num_object_clones":0,"num_object_copies":3015,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":8144,"num_write_kb":65950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8091,"ondisk_log_size":8091,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":535588,"kb_used_data":97920,"kb_used_omap":0,"kb_used_meta":437632,"kb_avail":376951772,"statfs":{"total":386547056640,"available":385998614528,"internally_reserved":0,"allocated":100270080,"data_stored":197845148,"data_compressed":10163658,"data_compressed_allocated":97701888,"data_compressed_original":196878336,"omap_allocated":0,"internal_metadata":448135168},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5414912,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":661,"num_write_kb":5288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000789"},"pg_stats":[{"pgid":"9.7","version":"72'632","reported_seq":669,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:00.331265+0000","last_change":"2024-04-23T01:56:03.683930+0000","last_active":"2024-04-23T01:57:00.331265+0000","last_peered":"2024-04-23T01:57:00.331265+0000","last_clean":"2024-04-23T01:57:00.331265+0000","last_became_active":"2024-04-23T01:56:03.675554+0000","last_became_peered":"2024-04-23T01:56:03.675554+0000","last_unstale":"2024-04-23T01:57:00.331265+0000","last_undegraded":"2024-04-23T01:57:00.331265+0000","last_fullsized":"2024-04-23T01:57:00.331265+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":5056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"74'585","reported_seq":622,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.668597+0000","last_change":"2024-04-23T01:56:03.552647+0000","last_active":"2024-04-23T01:57:01.668597+0000","last_peered":"2024-04-23T01:57:01.668597+0000","last_clean":"2024-04-23T01:57:01.668597+0000","last_became_active":"2024-04-23T01:56:03.541685+0000","last_became_peered":"2024-04-23T01:56:03.541685+0000","last_unstale":"2024-04-23T01:57:01.668597+0000","last_undegraded":"2024-04-23T01:57:01.668597+0000","last_fullsized":"2024-04-23T01:57:01.668597+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":585,"log_dups_size":0,"ondisk_log_size":585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4800512,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":586,"num_write_kb":4688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"72'503","reported_seq":538,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:59.401467+0000","last_change":"2024-04-23T01:56:03.515568+0000","last_active":"2024-04-23T01:56:59.401467+0000","last_peered":"2024-04-23T01:56:59.401467+0000","last_clean":"2024-04-23T01:56:59.401467+0000","last_became_active":"2024-04-23T01:56:03.510354+0000","last_became_peered":"2024-04-23T01:56:03.510354+0000","last_unstale":"2024-04-23T01:56:59.401467+0000","last_undegraded":"2024-04-23T01:56:59.401467+0000","last_fullsized":"2024-04-23T01:56:59.401467+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4128768,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":504,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.4","version":"72'495","reported_seq":530,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:57:00.066489+0000","last_change":"2024-04-23T01:56:03.647359+0000","last_active":"2024-04-23T01:57:00.066489+0000","last_peered":"2024-04-23T01:57:00.066489+0000","last_clean":"2024-04-23T01:57:00.066489+0000","last_became_active":"2024-04-23T01:56:03.642592+0000","last_became_peered":"2024-04-23T01:56:03.642592+0000","last_unstale":"2024-04-23T01:57:00.066489+0000","last_undegraded":"2024-04-23T01:57:00.066489+0000","last_fullsized":"2024-04-23T01:57:00.066489+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4063232,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":496,"num_write_kb":3968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"73'487","reported_seq":525,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.661737+0000","last_change":"2024-04-23T01:56:03.566150+0000","last_active":"2024-04-23T01:57:01.661737+0000","last_peered":"2024-04-23T01:57:01.661737+0000","last_clean":"2024-04-23T01:57:01.661737+0000","last_became_active":"2024-04-23T01:56:03.553086+0000","last_became_peered":"2024-04-23T01:56:03.553086+0000","last_unstale":"2024-04-23T01:57:01.661737+0000","last_undegraded":"2024-04-23T01:57:01.661737+0000","last_fullsized":"2024-04-23T01:57:01.661737+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3997696,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":488,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"74'388","reported_seq":425,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.652273+0000","last_change":"2024-04-23T01:56:03.605892+0000","last_active":"2024-04-23T01:57:01.652273+0000","last_peered":"2024-04-23T01:57:01.652273+0000","last_clean":"2024-04-23T01:57:01.652273+0000","last_became_active":"2024-04-23T01:56:03.601950+0000","last_became_peered":"2024-04-23T01:56:03.601950+0000","last_unstale":"2024-04-23T01:57:01.652273+0000","last_undegraded":"2024-04-23T01:57:01.652273+0000","last_fullsized":"2024-04-23T01:57:01.652273+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3186688,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":389,"num_write_kb":3112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"73'335","reported_seq":373,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.149652+0000","last_change":"2024-04-23T01:56:03.618761+0000","last_active":"2024-04-23T01:57:01.149652+0000","last_peered":"2024-04-23T01:57:01.149652+0000","last_clean":"2024-04-23T01:57:01.149652+0000","last_became_active":"2024-04-23T01:56:03.602521+0000","last_became_peered":"2024-04-23T01:56:03.602521+0000","last_unstale":"2024-04-23T01:57:01.149652+0000","last_undegraded":"2024-04-23T01:57:01.149652+0000","last_fullsized":"2024-04-23T01:57:01.149652+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"72'517","reported_seq":552,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.738746+0000","last_change":"2024-04-23T01:56:03.544149+0000","last_active":"2024-04-23T01:57:01.738746+0000","last_peered":"2024-04-23T01:57:01.738746+0000","last_clean":"2024-04-23T01:57:01.738746+0000","last_became_active":"2024-04-23T01:56:03.539589+0000","last_became_peered":"2024-04-23T01:56:03.539589+0000","last_unstale":"2024-04-23T01:57:01.738746+0000","last_undegraded":"2024-04-23T01:57:01.738746+0000","last_fullsized":"2024-04-23T01:57:01.738746+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4243456,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":518,"num_write_kb":4144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"73'625","reported_seq":663,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.714619+0000","last_change":"2024-04-23T01:56:03.635647+0000","last_active":"2024-04-23T01:57:01.714619+0000","last_peered":"2024-04-23T01:57:01.714619+0000","last_clean":"2024-04-23T01:57:01.714619+0000","last_became_active":"2024-04-23T01:56:03.627392+0000","last_became_peered":"2024-04-23T01:56:03.627392+0000","last_unstale":"2024-04-23T01:57:01.714619+0000","last_undegraded":"2024-04-23T01:57:01.714619+0000","last_fullsized":"2024-04-23T01:57:01.714619+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5128192,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":626,"num_write_kb":5008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":304,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:00.858838+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:57:00.858838+0000","last_peered":"2024-04-23T01:57:00.858838+0000","last_clean":"2024-04-23T01:57:00.858838+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:57:00.858838+0000","last_undegraded":"2024-04-23T01:57:00.858838+0000","last_fullsized":"2024-04-23T01:57:00.858838+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":172,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.357059+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:57:01.357059+0000","last_peered":"2024-04-23T01:57:01.357059+0000","last_clean":"2024-04-23T01:57:01.357059+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:57:01.357059+0000","last_undegraded":"2024-04-23T01:57:01.357059+0000","last_fullsized":"2024-04-23T01:57:01.357059+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.b","version":"72'456","reported_seq":493,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:00.349583+0000","last_change":"2024-04-23T01:56:03.687235+0000","last_active":"2024-04-23T01:57:00.349583+0000","last_peered":"2024-04-23T01:57:00.349583+0000","last_clean":"2024-04-23T01:57:00.349583+0000","last_became_active":"2024-04-23T01:56:03.683119+0000","last_became_peered":"2024-04-23T01:56:03.683119+0000","last_unstale":"2024-04-23T01:57:00.349583+0000","last_undegraded":"2024-04-23T01:57:00.349583+0000","last_fullsized":"2024-04-23T01:57:00.349583+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":456,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":165,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:00.348406+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:57:00.348406+0000","last_peered":"2024-04-23T01:57:00.348406+0000","last_clean":"2024-04-23T01:57:00.348406+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:57:00.348406+0000","last_undegraded":"2024-04-23T01:57:00.348406+0000","last_fullsized":"2024-04-23T01:57:00.348406+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"72'447","reported_seq":482,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:57:00.939516+0000","last_change":"2024-04-23T01:56:03.560440+0000","last_active":"2024-04-23T01:57:00.939516+0000","last_peered":"2024-04-23T01:57:00.939516+0000","last_clean":"2024-04-23T01:57:00.939516+0000","last_became_active":"2024-04-23T01:56:03.552840+0000","last_became_peered":"2024-04-23T01:56:03.552840+0000","last_unstale":"2024-04-23T01:57:00.939516+0000","last_undegraded":"2024-04-23T01:57:00.939516+0000","last_fullsized":"2024-04-23T01:57:00.939516+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":448,"num_write_kb":3584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":172,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.328035+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:57:01.328035+0000","last_peered":"2024-04-23T01:57:01.328035+0000","last_clean":"2024-04-23T01:57:01.328035+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:57:01.328035+0000","last_undegraded":"2024-04-23T01:57:01.328035+0000","last_fullsized":"2024-04-23T01:57:01.328035+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"72'488","reported_seq":525,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:00.854548+0000","last_change":"2024-04-23T01:56:03.597772+0000","last_active":"2024-04-23T01:57:00.854548+0000","last_peered":"2024-04-23T01:57:00.854548+0000","last_clean":"2024-04-23T01:57:00.854548+0000","last_became_active":"2024-04-23T01:56:03.589730+0000","last_became_peered":"2024-04-23T01:56:03.589730+0000","last_unstale":"2024-04-23T01:57:00.854548+0000","last_undegraded":"2024-04-23T01:57:00.854548+0000","last_fullsized":"2024-04-23T01:57:00.854548+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3997696,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":488,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":165,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:00.321885+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:57:00.321885+0000","last_peered":"2024-04-23T01:57:00.321885+0000","last_clean":"2024-04-23T01:57:00.321885+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:57:00.321885+0000","last_undegraded":"2024-04-23T01:57:00.321885+0000","last_fullsized":"2024-04-23T01:57:00.321885+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.f","version":"73'479","reported_seq":517,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:00.386073+0000","last_change":"2024-04-23T01:56:03.654933+0000","last_active":"2024-04-23T01:57:00.386073+0000","last_peered":"2024-04-23T01:57:00.386073+0000","last_clean":"2024-04-23T01:57:00.386073+0000","last_became_active":"2024-04-23T01:56:03.641185+0000","last_became_peered":"2024-04-23T01:56:03.641185+0000","last_unstale":"2024-04-23T01:57:00.386073+0000","last_undegraded":"2024-04-23T01:57:00.386073+0000","last_fullsized":"2024-04-23T01:57:00.386073+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":480,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":168,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:35.544714+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:56:35.544714+0000","last_peered":"2024-04-23T01:56:35.544714+0000","last_clean":"2024-04-23T01:56:35.544714+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:56:35.544714+0000","last_undegraded":"2024-04-23T01:56:35.544714+0000","last_fullsized":"2024-04-23T01:56:35.544714+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.e","version":"73'511","reported_seq":549,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.033071+0000","last_change":"2024-04-23T01:56:03.625197+0000","last_active":"2024-04-23T01:57:01.033071+0000","last_peered":"2024-04-23T01:57:01.033071+0000","last_clean":"2024-04-23T01:57:01.033071+0000","last_became_active":"2024-04-23T01:56:03.611498+0000","last_became_peered":"2024-04-23T01:56:03.611498+0000","last_unstale":"2024-04-23T01:57:01.033071+0000","last_undegraded":"2024-04-23T01:57:01.033071+0000","last_fullsized":"2024-04-23T01:57:01.033071+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":512,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":171,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:00.867573+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:57:00.867573+0000","last_peered":"2024-04-23T01:57:00.867573+0000","last_clean":"2024-04-23T01:57:00.867573+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:57:00.867573+0000","last_undegraded":"2024-04-23T01:57:00.867573+0000","last_fullsized":"2024-04-23T01:57:00.867573+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"73'591","reported_seq":629,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.537606+0000","last_change":"2024-04-23T01:56:03.612020+0000","last_active":"2024-04-23T01:57:01.537606+0000","last_peered":"2024-04-23T01:57:01.537606+0000","last_clean":"2024-04-23T01:57:01.537606+0000","last_became_active":"2024-04-23T01:56:03.605401+0000","last_became_peered":"2024-04-23T01:56:03.605401+0000","last_unstale":"2024-04-23T01:57:01.537606+0000","last_undegraded":"2024-04-23T01:57:01.537606+0000","last_fullsized":"2024-04-23T01:57:01.537606+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":592,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":168,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-23T01:56:35.552765+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:56:35.552765+0000","last_peered":"2024-04-23T01:56:35.552765+0000","last_clean":"2024-04-23T01:56:35.552765+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:56:35.552765+0000","last_undegraded":"2024-04-23T01:56:35.552765+0000","last_fullsized":"2024-04-23T01:56:35.552765+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.c","version":"74'455","reported_seq":492,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.625172+0000","last_change":"2024-04-23T01:56:03.574136+0000","last_active":"2024-04-23T01:57:01.625172+0000","last_peered":"2024-04-23T01:57:01.625172+0000","last_clean":"2024-04-23T01:57:01.625172+0000","last_became_active":"2024-04-23T01:56:03.566135+0000","last_became_peered":"2024-04-23T01:56:03.566135+0000","last_unstale":"2024-04-23T01:57:01.625172+0000","last_undegraded":"2024-04-23T01:57:01.625172+0000","last_fullsized":"2024-04-23T01:57:01.625172+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":456,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":172,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-23T01:57:01.354076+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:57:01.354076+0000","last_peered":"2024-04-23T01:57:01.354076+0000","last_clean":"2024-04-23T01:57:01.354076+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:57:01.354076+0000","last_undegraded":"2024-04-23T01:57:01.354076+0000","last_fullsized":"2024-04-23T01:57:01.354076+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":65593344,"num_objects":1003,"num_object_clones":0,"num_object_copies":3009,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8007,"num_write_kb":64056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":98365440,"data_stored":196730880,"data_compressed":10155015,"data_compressed_allocated":98365440,"data_compressed_original":196730880,"omap_allocated":0,"internal_metadata":0},"log_size":7994,"ondisk_log_size":7994,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706378,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":134300,"kb_used_data":26964,"kb_used_omap":0,"kb_used_meta":107328,"kb_avail":94237540,"statfs":{"total":96636764160,"available":96499240960,"internally_reserved":0,"allocated":27611136,"data_stored":54665391,"data_compressed":2809414,"data_compressed_allocated":26963968,"data_compressed_original":54419456,"omap_allocated":0,"internal_metadata":109903872},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706378,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":127932,"kb_used_data":20592,"kb_used_omap":0,"kb_used_meta":107328,"kb_avail":94243908,"statfs":{"total":96636764160,"available":96505761792,"internally_reserved":0,"allocated":21086208,"data_stored":41628674,"data_compressed":2136639,"data_compressed_allocated":20447232,"data_compressed_original":41385984,"omap_allocated":0,"internal_metadata":109903872},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706378,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":132020,"kb_used_data":20524,"kb_used_omap":0,"kb_used_meta":111488,"kb_avail":94239820,"statfs":{"total":96636764160,"available":96501575680,"internally_reserved":0,"allocated":21016576,"data_stored":40995644,"data_compressed":2104150,"data_compressed_allocated":20381696,"data_compressed_original":40763392,"omap_allocated":0,"internal_metadata":114163712},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706377,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":141336,"kb_used_data":29840,"kb_used_omap":0,"kb_used_meta":111488,"kb_avail":94230504,"statfs":{"total":96636764160,"available":96492036096,"internally_reserved":0,"allocated":30556160,"data_stored":60555439,"data_compressed":3113455,"data_compressed_allocated":29908992,"data_compressed_original":60309504,"omap_allocated":0,"internal_metadata":114163712},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":30363648,"data_stored":60727296,"data_compressed":3134685,"data_compressed_allocated":30363648,"data_compressed_original":60727296,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20422656,"data_stored":40845312,"data_compressed":2108380,"data_compressed_allocated":20422656,"data_compressed_original":40845312,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20586496,"data_stored":41172992,"data_compressed":2125298,"data_compressed_allocated":20586496,"data_compressed_original":41172992,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":26992640,"data_stored":53985280,"data_compressed":2786652,"data_compressed_allocated":26992640,"data_compressed_original":53985280,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:57:04.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:04.652+0000 7f584b22d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:04.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:04.660+0000 7f584b22d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:05.480 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 61 16 8513 8497 1.08815 1.02344 0.167639 0.114739 2024-04-23T01:57:06.480 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 62 16 8634 8618 1.08584 0.945312 0.112479 0.114942 2024-04-23T01:57:06.910 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:57:07.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:07.025+0000 7f2d9d624640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:07.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:07.032+0000 7f2d9d624640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:07.480 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 63 16 8778 8762 1.08646 1.125 0.0528959 0.1149 2024-04-23T01:57:08.480 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 64 16 8911 8895 1.08572 1.03906 0.048912 0.115005 2024-04-23T01:57:09.325 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:57:09.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:09.467+0000 7f07efc17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:09.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:09.474+0000 7f07efc17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:09.480 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 65 16 9027 9011 1.08296 0.90625 0.0980642 0.115284 2024-04-23T01:57:09.538 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:57:09.538 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:57:09.538 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:57:09.538 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:57:09.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:09.662+0000 7f88dc62d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:09.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:09.672+0000 7f88dc62d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:09.719 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:57:09.720 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:57:09.720 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:57:09.720 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:57:09.720 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:57:09.720 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:57:09.720 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:57:09.720 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:57:09.720 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:57:09.720 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:57:09.720 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:57:09.720 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:57:09.723 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:57:09.724 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:57:09.724 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:57:09.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:09.874+0000 7f6a96c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:09.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:09.884+0000 7f6a96c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:09.919 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:57:09.919 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:57:09.919 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:57:09.919 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:57:09.919 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:57:09.919 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:57:09.919 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:57:09.919 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:57:09.919 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:57:09.919 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:57:09.919 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:57:09.919 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:57:09.922 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:57:09.922 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:57:09.922 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:57:10.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:10.042+0000 7f9d9b42d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:10.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:10.054+0000 7f9d9b42d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:10.083 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:57:10.083 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:57:10.083 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:57:10.083 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:57:10.083 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:57:10.083 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:57:10.083 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:57:10.083 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:57:10.083 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:57:10.083 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:57:10.083 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:57:10.084 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:57:10.086 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:57:10.086 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:57:10.086 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:57:10.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:10.234+0000 7fb7d4ead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:10.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:10.247+0000 7fb7d4ead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:10.286 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:57:10.286 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:57:10.286 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:57:10.286 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:57:10.286 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:57:10.286 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:57:10.286 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:57:10.286 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:57:10.286 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:57:10.287 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:57:10.287 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:57:10.287 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:57:10.290 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:57:10.290 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:57:10.481 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 66 16 9155 9139 1.0817 1 0.0954683 0.115455 2024-04-23T01:57:11.481 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 67 16 9284 9268 1.0806 1.00781 0.098334 0.115539 2024-04-23T01:57:11.662 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:57:11.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:11.794+0000 7f817a19d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:11.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:11.801+0000 7f817a19d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:12.481 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 68 16 9414 9398 1.07964 1.01562 0.03985 0.115639 2024-04-23T01:57:13.481 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 69 16 9541 9525 1.07837 0.992188 0.133054 0.115816 2024-04-23T01:57:14.048 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:57:14.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:14.167+0000 7f970179d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:14.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:14.173+0000 7f970179d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:14.481 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 70 16 9666 9650 1.07691 0.976562 0.141136 0.115939 2024-04-23T01:57:15.291 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:57:15.291 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-23T01:57:15.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:15.402+0000 7f1d41a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:15.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:15.415+0000 7f1d41a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:15.481 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 71 16 9804 9788 1.07693 1.07812 0.0780685 0.115965 2024-04-23T01:57:16.330 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.0 to 1 (10000) 2024-04-23T01:57:16.348 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-23T01:57:16.373 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:57:16.481 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 72 16 9922 9906 1.07478 0.921875 0.194533 0.116158 2024-04-23T01:57:16.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:16.474+0000 7f151a86d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:16.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:16.488+0000 7f151a86d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:16.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:16.523+0000 7fcc3c95d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:16.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:16.531+0000 7fcc3c95d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:17.341 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.1 to 1 (10000) 2024-04-23T01:57:17.360 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-23T01:57:17.481 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 73 16 10058 10042 1.07461 1.0625 0.122601 0.1162 2024-04-23T01:57:17.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:17.491+0000 7fe958b9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:17.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:17.503+0000 7fe958b9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:18.344 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.2 to 1 (10000) 2024-04-23T01:57:18.360 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-23T01:57:18.481 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 74 16 10167 10151 1.07159 0.851562 0.24285 0.116455 2024-04-23T01:57:18.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:18.481+0000 7fcb3f864640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:18.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:18.492+0000 7fcb3f864640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:18.860 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:57:18.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:18.991+0000 7f84a20ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:19.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:18.998+0000 7f84a20ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:19.238 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.3 to 1 (10000) 2024-04-23T01:57:19.253 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-23T01:57:19.254 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:57:19.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:19.364+0000 7f21efcad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:19.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:19.374+0000 7f21efcad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:19.481 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 75 16 10270 10254 1.06803 0.804688 0.0440408 0.116884 2024-04-23T01:57:19.578 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:57:19.579 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:57:19.594 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":439,"stamp":"2024-04-23T01:57:19.237555+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":82027040,"num_objects":1247,"num_object_clones":0,"num_object_copies":3741,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":10078,"num_write_kb":81422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10023,"ondisk_log_size":10023,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":564576,"kb_used_data":121344,"kb_used_omap":0,"kb_used_meta":443200,"kb_avail":376922784,"statfs":{"total":386547056640,"available":385968930816,"internally_reserved":0,"allocated":124256256,"data_stored":245646693,"data_compressed":12629748,"data_compressed_allocated":121581568,"data_compressed_original":244637696,"omap_allocated":0,"internal_metadata":453836800},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5251072,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":641,"num_write_kb":5128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.547688"},"pg_stats":[{"pgid":"9.7","version":"76'711","reported_seq":756,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:16.448031+0000","last_change":"2024-04-23T01:56:03.683930+0000","last_active":"2024-04-23T01:57:16.448031+0000","last_peered":"2024-04-23T01:57:16.448031+0000","last_clean":"2024-04-23T01:57:16.448031+0000","last_became_active":"2024-04-23T01:56:03.675554+0000","last_became_peered":"2024-04-23T01:56:03.675554+0000","last_unstale":"2024-04-23T01:57:16.448031+0000","last_undegraded":"2024-04-23T01:57:16.448031+0000","last_fullsized":"2024-04-23T01:57:16.448031+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5832704,"num_objects":89,"num_object_clones":0,"num_object_copies":267,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":712,"num_write_kb":5696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"76'712","reported_seq":757,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-23T01:57:16.379780+0000","last_change":"2024-04-23T01:56:03.552647+0000","last_active":"2024-04-23T01:57:16.379780+0000","last_peered":"2024-04-23T01:57:16.379780+0000","last_clean":"2024-04-23T01:57:16.379780+0000","last_became_active":"2024-04-23T01:56:03.541685+0000","last_became_peered":"2024-04-23T01:56:03.541685+0000","last_unstale":"2024-04-23T01:57:16.379780+0000","last_undegraded":"2024-04-23T01:57:16.379780+0000","last_fullsized":"2024-04-23T01:57:16.379780+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5832704,"num_objects":89,"num_object_clones":0,"num_object_copies":267,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":712,"num_write_kb":5696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"76'603","reported_seq":646,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:16.727175+0000","last_change":"2024-04-23T01:56:03.515568+0000","last_active":"2024-04-23T01:57:16.727175+0000","last_peered":"2024-04-23T01:57:16.727175+0000","last_clean":"2024-04-23T01:57:16.727175+0000","last_became_active":"2024-04-23T01:56:03.510354+0000","last_became_peered":"2024-04-23T01:56:03.510354+0000","last_unstale":"2024-04-23T01:57:16.727175+0000","last_undegraded":"2024-04-23T01:57:16.727175+0000","last_fullsized":"2024-04-23T01:57:16.727175+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4947968,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":604,"num_write_kb":4832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.4","version":"76'695","reported_seq":738,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:15.436594+0000","last_change":"2024-04-23T01:56:03.647359+0000","last_active":"2024-04-23T01:57:15.436594+0000","last_peered":"2024-04-23T01:57:15.436594+0000","last_clean":"2024-04-23T01:57:15.436594+0000","last_became_active":"2024-04-23T01:56:03.642592+0000","last_became_peered":"2024-04-23T01:56:03.642592+0000","last_unstale":"2024-04-23T01:57:15.436594+0000","last_undegraded":"2024-04-23T01:57:15.436594+0000","last_fullsized":"2024-04-23T01:57:15.436594+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":696,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"76'590","reported_seq":635,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:16.692910+0000","last_change":"2024-04-23T01:56:03.566150+0000","last_active":"2024-04-23T01:57:16.692910+0000","last_peered":"2024-04-23T01:57:16.692910+0000","last_clean":"2024-04-23T01:57:16.692910+0000","last_became_active":"2024-04-23T01:56:03.553086+0000","last_became_peered":"2024-04-23T01:56:03.553086+0000","last_unstale":"2024-04-23T01:57:16.692910+0000","last_undegraded":"2024-04-23T01:57:16.692910+0000","last_fullsized":"2024-04-23T01:57:16.692910+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4841472,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":591,"num_write_kb":4728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"77'503","reported_seq":549,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-23T01:57:16.546634+0000","last_change":"2024-04-23T01:56:03.605892+0000","last_active":"2024-04-23T01:57:16.546634+0000","last_peered":"2024-04-23T01:57:16.546634+0000","last_clean":"2024-04-23T01:57:16.546634+0000","last_became_active":"2024-04-23T01:56:03.601950+0000","last_became_peered":"2024-04-23T01:56:03.601950+0000","last_unstale":"2024-04-23T01:57:16.546634+0000","last_undegraded":"2024-04-23T01:57:16.546634+0000","last_fullsized":"2024-04-23T01:57:16.546634+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4128768,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":504,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"76'471","reported_seq":516,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:15.917058+0000","last_change":"2024-04-23T01:56:03.618761+0000","last_active":"2024-04-23T01:57:15.917058+0000","last_peered":"2024-04-23T01:57:15.917058+0000","last_clean":"2024-04-23T01:57:15.917058+0000","last_became_active":"2024-04-23T01:56:03.602521+0000","last_became_peered":"2024-04-23T01:56:03.602521+0000","last_unstale":"2024-04-23T01:57:15.917058+0000","last_undegraded":"2024-04-23T01:57:15.917058+0000","last_fullsized":"2024-04-23T01:57:15.917058+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3866624,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":472,"num_write_kb":3776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"76'615","reported_seq":658,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:15.504957+0000","last_change":"2024-04-23T01:56:03.544149+0000","last_active":"2024-04-23T01:57:15.504957+0000","last_peered":"2024-04-23T01:57:15.504957+0000","last_clean":"2024-04-23T01:57:15.504957+0000","last_became_active":"2024-04-23T01:56:03.539589+0000","last_became_peered":"2024-04-23T01:56:03.539589+0000","last_unstale":"2024-04-23T01:57:15.504957+0000","last_undegraded":"2024-04-23T01:57:15.504957+0000","last_fullsized":"2024-04-23T01:57:15.504957+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5046272,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":616,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"76'751","reported_seq":796,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:16.366910+0000","last_change":"2024-04-23T01:56:03.635647+0000","last_active":"2024-04-23T01:57:16.366910+0000","last_peered":"2024-04-23T01:57:16.366910+0000","last_clean":"2024-04-23T01:57:16.366910+0000","last_became_active":"2024-04-23T01:56:03.627392+0000","last_became_peered":"2024-04-23T01:56:03.627392+0000","last_unstale":"2024-04-23T01:57:16.366910+0000","last_undegraded":"2024-04-23T01:57:16.366910+0000","last_fullsized":"2024-04-23T01:57:16.366910+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":311,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:05.683902+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:57:05.683902+0000","last_peered":"2024-04-23T01:57:05.683902+0000","last_clean":"2024-04-23T01:57:05.683902+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:57:05.683902+0000","last_undegraded":"2024-04-23T01:57:05.683902+0000","last_fullsized":"2024-04-23T01:57:05.683902+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":181,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-23T01:57:16.378199+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:57:16.378199+0000","last_peered":"2024-04-23T01:57:16.378199+0000","last_clean":"2024-04-23T01:57:16.378199+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:57:16.378199+0000","last_undegraded":"2024-04-23T01:57:16.378199+0000","last_fullsized":"2024-04-23T01:57:16.378199+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.b","version":"76'551","reported_seq":596,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:14.641088+0000","last_change":"2024-04-23T01:56:03.687235+0000","last_active":"2024-04-23T01:57:14.641088+0000","last_peered":"2024-04-23T01:57:14.641088+0000","last_clean":"2024-04-23T01:57:14.641088+0000","last_became_active":"2024-04-23T01:56:03.683119+0000","last_became_peered":"2024-04-23T01:56:03.683119+0000","last_unstale":"2024-04-23T01:57:14.641088+0000","last_undegraded":"2024-04-23T01:57:14.641088+0000","last_fullsized":"2024-04-23T01:57:14.641088+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4521984,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":552,"num_write_kb":4416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":172,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:05.687040+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:57:05.687040+0000","last_peered":"2024-04-23T01:57:05.687040+0000","last_clean":"2024-04-23T01:57:05.687040+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:57:05.687040+0000","last_undegraded":"2024-04-23T01:57:05.687040+0000","last_fullsized":"2024-04-23T01:57:05.687040+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"76'583","reported_seq":626,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:16.167805+0000","last_change":"2024-04-23T01:56:03.560440+0000","last_active":"2024-04-23T01:57:16.167805+0000","last_peered":"2024-04-23T01:57:16.167805+0000","last_clean":"2024-04-23T01:57:16.167805+0000","last_became_active":"2024-04-23T01:56:03.552840+0000","last_became_peered":"2024-04-23T01:56:03.552840+0000","last_unstale":"2024-04-23T01:57:16.167805+0000","last_undegraded":"2024-04-23T01:57:16.167805+0000","last_fullsized":"2024-04-23T01:57:16.167805+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4784128,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":584,"num_write_kb":4672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":181,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-23T01:57:16.370363+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:57:16.370363+0000","last_peered":"2024-04-23T01:57:16.370363+0000","last_clean":"2024-04-23T01:57:16.370363+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:57:16.370363+0000","last_undegraded":"2024-04-23T01:57:16.370363+0000","last_fullsized":"2024-04-23T01:57:16.370363+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"76'592","reported_seq":637,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:13.624166+0000","last_change":"2024-04-23T01:56:03.597772+0000","last_active":"2024-04-23T01:57:13.624166+0000","last_peered":"2024-04-23T01:57:13.624166+0000","last_clean":"2024-04-23T01:57:13.624166+0000","last_became_active":"2024-04-23T01:56:03.589730+0000","last_became_peered":"2024-04-23T01:56:03.589730+0000","last_unstale":"2024-04-23T01:57:13.624166+0000","last_undegraded":"2024-04-23T01:57:13.624166+0000","last_fullsized":"2024-04-23T01:57:13.624166+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4857856,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":593,"num_write_kb":4744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":172,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:05.691914+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:57:05.691914+0000","last_peered":"2024-04-23T01:57:05.691914+0000","last_clean":"2024-04-23T01:57:05.691914+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:57:05.691914+0000","last_undegraded":"2024-04-23T01:57:05.691914+0000","last_fullsized":"2024-04-23T01:57:05.691914+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.f","version":"76'615","reported_seq":660,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:16.092920+0000","last_change":"2024-04-23T01:56:03.654933+0000","last_active":"2024-04-23T01:57:16.092920+0000","last_peered":"2024-04-23T01:57:16.092920+0000","last_clean":"2024-04-23T01:57:16.092920+0000","last_became_active":"2024-04-23T01:56:03.641185+0000","last_became_peered":"2024-04-23T01:56:03.641185+0000","last_unstale":"2024-04-23T01:57:16.092920+0000","last_undegraded":"2024-04-23T01:57:16.092920+0000","last_fullsized":"2024-04-23T01:57:16.092920+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5046272,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":616,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":176,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:05.686147+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:57:05.686147+0000","last_peered":"2024-04-23T01:57:05.686147+0000","last_clean":"2024-04-23T01:57:05.686147+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:57:05.686147+0000","last_undegraded":"2024-04-23T01:57:05.686147+0000","last_fullsized":"2024-04-23T01:57:05.686147+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.e","version":"76'616","reported_seq":661,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:16.636118+0000","last_change":"2024-04-23T01:56:03.625197+0000","last_active":"2024-04-23T01:57:16.636118+0000","last_peered":"2024-04-23T01:57:16.636118+0000","last_clean":"2024-04-23T01:57:16.636118+0000","last_became_active":"2024-04-23T01:56:03.611498+0000","last_became_peered":"2024-04-23T01:56:03.611498+0000","last_unstale":"2024-04-23T01:57:16.636118+0000","last_undegraded":"2024-04-23T01:57:16.636118+0000","last_fullsized":"2024-04-23T01:57:16.636118+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5054464,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":617,"num_write_kb":4936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":178,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:05.718423+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:57:05.718423+0000","last_peered":"2024-04-23T01:57:05.718423+0000","last_clean":"2024-04-23T01:57:05.718423+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:57:05.718423+0000","last_undegraded":"2024-04-23T01:57:05.718423+0000","last_fullsized":"2024-04-23T01:57:05.718423+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"76'727","reported_seq":772,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:16.232014+0000","last_change":"2024-04-23T01:56:03.612020+0000","last_active":"2024-04-23T01:57:16.232014+0000","last_peered":"2024-04-23T01:57:16.232014+0000","last_clean":"2024-04-23T01:57:16.232014+0000","last_became_active":"2024-04-23T01:56:03.605401+0000","last_became_peered":"2024-04-23T01:56:03.605401+0000","last_unstale":"2024-04-23T01:57:16.232014+0000","last_undegraded":"2024-04-23T01:57:16.232014+0000","last_fullsized":"2024-04-23T01:57:16.232014+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":728,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":176,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:05.699736+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:57:05.699736+0000","last_peered":"2024-04-23T01:57:05.699736+0000","last_clean":"2024-04-23T01:57:05.699736+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:57:05.699736+0000","last_undegraded":"2024-04-23T01:57:05.699736+0000","last_fullsized":"2024-04-23T01:57:05.699736+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.c","version":"76'591","reported_seq":637,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-23T01:57:16.370797+0000","last_change":"2024-04-23T01:56:03.574136+0000","last_active":"2024-04-23T01:57:16.370797+0000","last_peered":"2024-04-23T01:57:16.370797+0000","last_clean":"2024-04-23T01:57:16.370797+0000","last_became_active":"2024-04-23T01:56:03.566135+0000","last_became_peered":"2024-04-23T01:56:03.566135+0000","last_unstale":"2024-04-23T01:57:16.370797+0000","last_undegraded":"2024-04-23T01:57:16.370797+0000","last_fullsized":"2024-04-23T01:57:16.370797+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":592,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":181,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-23T01:57:16.376715+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:57:16.376715+0000","last_peered":"2024-04-23T01:57:16.376715+0000","last_clean":"2024-04-23T01:57:16.376715+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:57:16.376715+0000","last_undegraded":"2024-04-23T01:57:16.376715+0000","last_fullsized":"2024-04-23T01:57:16.376715+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":81436672,"num_objects":1245,"num_object_clones":0,"num_object_copies":3735,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9941,"num_write_kb":79528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122134528,"data_stored":244269056,"data_compressed":12609684,"data_compressed_allocated":122134528,"data_compressed_original":244269056,"omap_allocated":0,"internal_metadata":0},"log_size":9926,"ondisk_log_size":9926,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706393,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":163008,"kb_used_data":33464,"kb_used_omap":0,"kb_used_meta":129536,"kb_avail":94208832,"statfs":{"total":96636764160,"available":96469843968,"internally_reserved":0,"allocated":34267136,"data_stored":67937974,"data_compressed":3494251,"data_compressed_allocated":33595392,"data_compressed_original":67682304,"omap_allocated":0,"internal_metadata":132644864},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706393,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":137040,"kb_used_data":25544,"kb_used_omap":0,"kb_used_meta":111488,"kb_avail":94234800,"statfs":{"total":96636764160,"available":96496435200,"internally_reserved":0,"allocated":26157056,"data_stored":51717814,"data_compressed":2656929,"data_compressed_allocated":25485312,"data_compressed_original":51462144,"omap_allocated":0,"internal_metadata":114163712},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706393,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":124232,"kb_used_data":25216,"kb_used_omap":0,"kb_used_meta":99008,"kb_avail":94247608,"statfs":{"total":96636764160,"available":96509550592,"internally_reserved":0,"allocated":25821184,"data_stored":50565443,"data_compressed":2597791,"data_compressed_allocated":25161728,"data_compressed_original":50323456,"omap_allocated":0,"internal_metadata":101384192},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706392,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":140296,"kb_used_data":37120,"kb_used_omap":0,"kb_used_meta":103168,"kb_avail":94231544,"statfs":{"total":96636764160,"available":96493101056,"internally_reserved":0,"allocated":38010880,"data_stored":75425462,"data_compressed":3880777,"data_compressed_allocated":37339136,"data_compressed_original":75169792,"omap_allocated":0,"internal_metadata":105644032},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":37769216,"data_stored":75538432,"data_compressed":3899469,"data_compressed_allocated":37769216,"data_compressed_original":75538432,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":25169920,"data_stored":50339840,"data_compressed":2598637,"data_compressed_allocated":25169920,"data_compressed_original":50339840,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":25595904,"data_stored":51191808,"data_compressed":2642627,"data_compressed_allocated":25595904,"data_compressed_original":51191808,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":33599488,"data_stored":67198976,"data_compressed":3468951,"data_compressed_allocated":33599488,"data_compressed_original":67198976,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:57:19.595 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:57:19.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:19.708+0000 7f1e088ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:19.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:19.717+0000 7f1e088ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:19.922 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:57:19.922 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:57:19.936 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":440,"stamp":"2024-04-23T01:57:19.691047+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":82027040,"num_objects":1247,"num_object_clones":0,"num_object_copies":3741,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":10078,"num_write_kb":81422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10023,"ondisk_log_size":10023,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":564576,"kb_used_data":121344,"kb_used_omap":0,"kb_used_meta":443200,"kb_avail":376922784,"statfs":{"total":386547056640,"available":385968930816,"internally_reserved":0,"allocated":124256256,"data_stored":245646693,"data_compressed":12629748,"data_compressed_allocated":121581568,"data_compressed_original":244637696,"omap_allocated":0,"internal_metadata":453836800},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5251072,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":641,"num_write_kb":5128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000659"},"pg_stats":[{"pgid":"9.7","version":"76'711","reported_seq":756,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:16.448031+0000","last_change":"2024-04-23T01:56:03.683930+0000","last_active":"2024-04-23T01:57:16.448031+0000","last_peered":"2024-04-23T01:57:16.448031+0000","last_clean":"2024-04-23T01:57:16.448031+0000","last_became_active":"2024-04-23T01:56:03.675554+0000","last_became_peered":"2024-04-23T01:56:03.675554+0000","last_unstale":"2024-04-23T01:57:16.448031+0000","last_undegraded":"2024-04-23T01:57:16.448031+0000","last_fullsized":"2024-04-23T01:57:16.448031+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5832704,"num_objects":89,"num_object_clones":0,"num_object_copies":267,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":712,"num_write_kb":5696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"76'712","reported_seq":757,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-23T01:57:16.379780+0000","last_change":"2024-04-23T01:56:03.552647+0000","last_active":"2024-04-23T01:57:16.379780+0000","last_peered":"2024-04-23T01:57:16.379780+0000","last_clean":"2024-04-23T01:57:16.379780+0000","last_became_active":"2024-04-23T01:56:03.541685+0000","last_became_peered":"2024-04-23T01:56:03.541685+0000","last_unstale":"2024-04-23T01:57:16.379780+0000","last_undegraded":"2024-04-23T01:57:16.379780+0000","last_fullsized":"2024-04-23T01:57:16.379780+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5832704,"num_objects":89,"num_object_clones":0,"num_object_copies":267,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":712,"num_write_kb":5696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"76'603","reported_seq":646,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:16.727175+0000","last_change":"2024-04-23T01:56:03.515568+0000","last_active":"2024-04-23T01:57:16.727175+0000","last_peered":"2024-04-23T01:57:16.727175+0000","last_clean":"2024-04-23T01:57:16.727175+0000","last_became_active":"2024-04-23T01:56:03.510354+0000","last_became_peered":"2024-04-23T01:56:03.510354+0000","last_unstale":"2024-04-23T01:57:16.727175+0000","last_undegraded":"2024-04-23T01:57:16.727175+0000","last_fullsized":"2024-04-23T01:57:16.727175+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4947968,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":604,"num_write_kb":4832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.4","version":"76'695","reported_seq":738,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:15.436594+0000","last_change":"2024-04-23T01:56:03.647359+0000","last_active":"2024-04-23T01:57:15.436594+0000","last_peered":"2024-04-23T01:57:15.436594+0000","last_clean":"2024-04-23T01:57:15.436594+0000","last_became_active":"2024-04-23T01:56:03.642592+0000","last_became_peered":"2024-04-23T01:56:03.642592+0000","last_unstale":"2024-04-23T01:57:15.436594+0000","last_undegraded":"2024-04-23T01:57:15.436594+0000","last_fullsized":"2024-04-23T01:57:15.436594+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":696,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"76'590","reported_seq":635,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:16.692910+0000","last_change":"2024-04-23T01:56:03.566150+0000","last_active":"2024-04-23T01:57:16.692910+0000","last_peered":"2024-04-23T01:57:16.692910+0000","last_clean":"2024-04-23T01:57:16.692910+0000","last_became_active":"2024-04-23T01:56:03.553086+0000","last_became_peered":"2024-04-23T01:56:03.553086+0000","last_unstale":"2024-04-23T01:57:16.692910+0000","last_undegraded":"2024-04-23T01:57:16.692910+0000","last_fullsized":"2024-04-23T01:57:16.692910+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4841472,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":591,"num_write_kb":4728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"77'503","reported_seq":549,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-23T01:57:16.546634+0000","last_change":"2024-04-23T01:56:03.605892+0000","last_active":"2024-04-23T01:57:16.546634+0000","last_peered":"2024-04-23T01:57:16.546634+0000","last_clean":"2024-04-23T01:57:16.546634+0000","last_became_active":"2024-04-23T01:56:03.601950+0000","last_became_peered":"2024-04-23T01:56:03.601950+0000","last_unstale":"2024-04-23T01:57:16.546634+0000","last_undegraded":"2024-04-23T01:57:16.546634+0000","last_fullsized":"2024-04-23T01:57:16.546634+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4128768,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":504,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"76'471","reported_seq":516,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:15.917058+0000","last_change":"2024-04-23T01:56:03.618761+0000","last_active":"2024-04-23T01:57:15.917058+0000","last_peered":"2024-04-23T01:57:15.917058+0000","last_clean":"2024-04-23T01:57:15.917058+0000","last_became_active":"2024-04-23T01:56:03.602521+0000","last_became_peered":"2024-04-23T01:56:03.602521+0000","last_unstale":"2024-04-23T01:57:15.917058+0000","last_undegraded":"2024-04-23T01:57:15.917058+0000","last_fullsized":"2024-04-23T01:57:15.917058+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3866624,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":472,"num_write_kb":3776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"76'615","reported_seq":658,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:15.504957+0000","last_change":"2024-04-23T01:56:03.544149+0000","last_active":"2024-04-23T01:57:15.504957+0000","last_peered":"2024-04-23T01:57:15.504957+0000","last_clean":"2024-04-23T01:57:15.504957+0000","last_became_active":"2024-04-23T01:56:03.539589+0000","last_became_peered":"2024-04-23T01:56:03.539589+0000","last_unstale":"2024-04-23T01:57:15.504957+0000","last_undegraded":"2024-04-23T01:57:15.504957+0000","last_fullsized":"2024-04-23T01:57:15.504957+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5046272,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":616,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"76'751","reported_seq":796,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:16.366910+0000","last_change":"2024-04-23T01:56:03.635647+0000","last_active":"2024-04-23T01:57:16.366910+0000","last_peered":"2024-04-23T01:57:16.366910+0000","last_clean":"2024-04-23T01:57:16.366910+0000","last_became_active":"2024-04-23T01:56:03.627392+0000","last_became_peered":"2024-04-23T01:56:03.627392+0000","last_unstale":"2024-04-23T01:57:16.366910+0000","last_undegraded":"2024-04-23T01:57:16.366910+0000","last_fullsized":"2024-04-23T01:57:16.366910+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":311,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:05.683902+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:57:05.683902+0000","last_peered":"2024-04-23T01:57:05.683902+0000","last_clean":"2024-04-23T01:57:05.683902+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:57:05.683902+0000","last_undegraded":"2024-04-23T01:57:05.683902+0000","last_fullsized":"2024-04-23T01:57:05.683902+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":181,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-23T01:57:16.378199+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:57:16.378199+0000","last_peered":"2024-04-23T01:57:16.378199+0000","last_clean":"2024-04-23T01:57:16.378199+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:57:16.378199+0000","last_undegraded":"2024-04-23T01:57:16.378199+0000","last_fullsized":"2024-04-23T01:57:16.378199+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.b","version":"76'551","reported_seq":596,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:14.641088+0000","last_change":"2024-04-23T01:56:03.687235+0000","last_active":"2024-04-23T01:57:14.641088+0000","last_peered":"2024-04-23T01:57:14.641088+0000","last_clean":"2024-04-23T01:57:14.641088+0000","last_became_active":"2024-04-23T01:56:03.683119+0000","last_became_peered":"2024-04-23T01:56:03.683119+0000","last_unstale":"2024-04-23T01:57:14.641088+0000","last_undegraded":"2024-04-23T01:57:14.641088+0000","last_fullsized":"2024-04-23T01:57:14.641088+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4521984,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":552,"num_write_kb":4416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":172,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:05.687040+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:57:05.687040+0000","last_peered":"2024-04-23T01:57:05.687040+0000","last_clean":"2024-04-23T01:57:05.687040+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:57:05.687040+0000","last_undegraded":"2024-04-23T01:57:05.687040+0000","last_fullsized":"2024-04-23T01:57:05.687040+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"76'583","reported_seq":626,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:16.167805+0000","last_change":"2024-04-23T01:56:03.560440+0000","last_active":"2024-04-23T01:57:16.167805+0000","last_peered":"2024-04-23T01:57:16.167805+0000","last_clean":"2024-04-23T01:57:16.167805+0000","last_became_active":"2024-04-23T01:56:03.552840+0000","last_became_peered":"2024-04-23T01:56:03.552840+0000","last_unstale":"2024-04-23T01:57:16.167805+0000","last_undegraded":"2024-04-23T01:57:16.167805+0000","last_fullsized":"2024-04-23T01:57:16.167805+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4784128,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":584,"num_write_kb":4672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":181,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-23T01:57:16.370363+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:57:16.370363+0000","last_peered":"2024-04-23T01:57:16.370363+0000","last_clean":"2024-04-23T01:57:16.370363+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:57:16.370363+0000","last_undegraded":"2024-04-23T01:57:16.370363+0000","last_fullsized":"2024-04-23T01:57:16.370363+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"76'592","reported_seq":637,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:13.624166+0000","last_change":"2024-04-23T01:56:03.597772+0000","last_active":"2024-04-23T01:57:13.624166+0000","last_peered":"2024-04-23T01:57:13.624166+0000","last_clean":"2024-04-23T01:57:13.624166+0000","last_became_active":"2024-04-23T01:56:03.589730+0000","last_became_peered":"2024-04-23T01:56:03.589730+0000","last_unstale":"2024-04-23T01:57:13.624166+0000","last_undegraded":"2024-04-23T01:57:13.624166+0000","last_fullsized":"2024-04-23T01:57:13.624166+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4857856,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":593,"num_write_kb":4744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":172,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:05.691914+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:57:05.691914+0000","last_peered":"2024-04-23T01:57:05.691914+0000","last_clean":"2024-04-23T01:57:05.691914+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:57:05.691914+0000","last_undegraded":"2024-04-23T01:57:05.691914+0000","last_fullsized":"2024-04-23T01:57:05.691914+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.f","version":"76'615","reported_seq":660,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:16.092920+0000","last_change":"2024-04-23T01:56:03.654933+0000","last_active":"2024-04-23T01:57:16.092920+0000","last_peered":"2024-04-23T01:57:16.092920+0000","last_clean":"2024-04-23T01:57:16.092920+0000","last_became_active":"2024-04-23T01:56:03.641185+0000","last_became_peered":"2024-04-23T01:56:03.641185+0000","last_unstale":"2024-04-23T01:57:16.092920+0000","last_undegraded":"2024-04-23T01:57:16.092920+0000","last_fullsized":"2024-04-23T01:57:16.092920+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5046272,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":616,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":176,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:05.686147+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:57:05.686147+0000","last_peered":"2024-04-23T01:57:05.686147+0000","last_clean":"2024-04-23T01:57:05.686147+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:57:05.686147+0000","last_undegraded":"2024-04-23T01:57:05.686147+0000","last_fullsized":"2024-04-23T01:57:05.686147+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.e","version":"76'616","reported_seq":661,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:16.636118+0000","last_change":"2024-04-23T01:56:03.625197+0000","last_active":"2024-04-23T01:57:16.636118+0000","last_peered":"2024-04-23T01:57:16.636118+0000","last_clean":"2024-04-23T01:57:16.636118+0000","last_became_active":"2024-04-23T01:56:03.611498+0000","last_became_peered":"2024-04-23T01:56:03.611498+0000","last_unstale":"2024-04-23T01:57:16.636118+0000","last_undegraded":"2024-04-23T01:57:16.636118+0000","last_fullsized":"2024-04-23T01:57:16.636118+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5054464,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":617,"num_write_kb":4936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":178,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:05.718423+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:57:05.718423+0000","last_peered":"2024-04-23T01:57:05.718423+0000","last_clean":"2024-04-23T01:57:05.718423+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:57:05.718423+0000","last_undegraded":"2024-04-23T01:57:05.718423+0000","last_fullsized":"2024-04-23T01:57:05.718423+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"76'727","reported_seq":772,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:16.232014+0000","last_change":"2024-04-23T01:56:03.612020+0000","last_active":"2024-04-23T01:57:16.232014+0000","last_peered":"2024-04-23T01:57:16.232014+0000","last_clean":"2024-04-23T01:57:16.232014+0000","last_became_active":"2024-04-23T01:56:03.605401+0000","last_became_peered":"2024-04-23T01:56:03.605401+0000","last_unstale":"2024-04-23T01:57:16.232014+0000","last_undegraded":"2024-04-23T01:57:16.232014+0000","last_fullsized":"2024-04-23T01:57:16.232014+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":728,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":176,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-23T01:57:05.699736+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:57:05.699736+0000","last_peered":"2024-04-23T01:57:05.699736+0000","last_clean":"2024-04-23T01:57:05.699736+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:57:05.699736+0000","last_undegraded":"2024-04-23T01:57:05.699736+0000","last_fullsized":"2024-04-23T01:57:05.699736+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.c","version":"76'591","reported_seq":637,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-23T01:57:16.370797+0000","last_change":"2024-04-23T01:56:03.574136+0000","last_active":"2024-04-23T01:57:16.370797+0000","last_peered":"2024-04-23T01:57:16.370797+0000","last_clean":"2024-04-23T01:57:16.370797+0000","last_became_active":"2024-04-23T01:56:03.566135+0000","last_became_peered":"2024-04-23T01:56:03.566135+0000","last_unstale":"2024-04-23T01:57:16.370797+0000","last_undegraded":"2024-04-23T01:57:16.370797+0000","last_fullsized":"2024-04-23T01:57:16.370797+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":592,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":181,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-23T01:57:16.376715+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:57:16.376715+0000","last_peered":"2024-04-23T01:57:16.376715+0000","last_clean":"2024-04-23T01:57:16.376715+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:57:16.376715+0000","last_undegraded":"2024-04-23T01:57:16.376715+0000","last_fullsized":"2024-04-23T01:57:16.376715+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":81436672,"num_objects":1245,"num_object_clones":0,"num_object_copies":3735,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9941,"num_write_kb":79528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122134528,"data_stored":244269056,"data_compressed":12609684,"data_compressed_allocated":122134528,"data_compressed_original":244269056,"omap_allocated":0,"internal_metadata":0},"log_size":9926,"ondisk_log_size":9926,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706393,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":163008,"kb_used_data":33464,"kb_used_omap":0,"kb_used_meta":129536,"kb_avail":94208832,"statfs":{"total":96636764160,"available":96469843968,"internally_reserved":0,"allocated":34267136,"data_stored":67937974,"data_compressed":3494251,"data_compressed_allocated":33595392,"data_compressed_original":67682304,"omap_allocated":0,"internal_metadata":132644864},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706393,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":137040,"kb_used_data":25544,"kb_used_omap":0,"kb_used_meta":111488,"kb_avail":94234800,"statfs":{"total":96636764160,"available":96496435200,"internally_reserved":0,"allocated":26157056,"data_stored":51717814,"data_compressed":2656929,"data_compressed_allocated":25485312,"data_compressed_original":51462144,"omap_allocated":0,"internal_metadata":114163712},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706393,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":124232,"kb_used_data":25216,"kb_used_omap":0,"kb_used_meta":99008,"kb_avail":94247608,"statfs":{"total":96636764160,"available":96509550592,"internally_reserved":0,"allocated":25821184,"data_stored":50565443,"data_compressed":2597791,"data_compressed_allocated":25161728,"data_compressed_original":50323456,"omap_allocated":0,"internal_metadata":101384192},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706392,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":140296,"kb_used_data":37120,"kb_used_omap":0,"kb_used_meta":103168,"kb_avail":94231544,"statfs":{"total":96636764160,"available":96493101056,"internally_reserved":0,"allocated":38010880,"data_stored":75425462,"data_compressed":3880777,"data_compressed_allocated":37339136,"data_compressed_original":75169792,"omap_allocated":0,"internal_metadata":105644032},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":37769216,"data_stored":75538432,"data_compressed":3899469,"data_compressed_allocated":37769216,"data_compressed_original":75538432,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":25169920,"data_stored":50339840,"data_compressed":2598637,"data_compressed_allocated":25169920,"data_compressed_original":50339840,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":25595904,"data_stored":51191808,"data_compressed":2642627,"data_compressed_allocated":25595904,"data_compressed_original":51191808,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":33599488,"data_stored":67198976,"data_compressed":3468951,"data_compressed_allocated":33599488,"data_compressed_original":67198976,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:57:19.937 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-23T01:57:19.938 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:57:19.938 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:57:19.938 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:57:20.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:20.053+0000 7fb90a62d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:20.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:20.066+0000 7fb90a62d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:20.124 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:57:20.124 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:57:20.124 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:57:20.125 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:57:20.125 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:57:20.125 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:57:20.125 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:57:20.125 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:57:20.125 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:57:20.125 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:57:20.125 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:57:20.125 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:57:20.127 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:57:20.128 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:57:20.128 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:57:20.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:20.254+0000 7f0eaef9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:20.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:20.265+0000 7f0eaef9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:20.285 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:57:20.285 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:57:20.286 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:57:20.286 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:57:20.286 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:57:20.286 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:57:20.286 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:57:20.286 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:57:20.286 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:57:20.286 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:57:20.286 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:57:20.286 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:57:20.289 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:57:20.289 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:57:20.289 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:57:20.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:20.409+0000 7f2feafee640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:20.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:20.420+0000 7f2feafee640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:20.467 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:57:20.467 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:57:20.467 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:57:20.467 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:57:20.467 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:57:20.467 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:57:20.468 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:57:20.468 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:57:20.468 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:57:20.468 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:57:20.468 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:57:20.468 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:57:20.469 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:57:20.470 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:57:20.470 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:57:20.481 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 76 16 10400 10384 1.06734 1.01562 0.172878 0.117011 2024-04-23T01:57:20.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:20.580+0000 7f7649217640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:20.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:20.589+0000 7f7649217640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:20.653 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:57:20.653 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:57:20.654 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:57:20.654 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:57:20.654 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:57:20.654 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:57:20.654 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:57:20.654 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:57:20.654 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:57:20.654 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:57:20.654 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:57:20.654 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:57:20.656 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:57:20.656 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:57:21.186 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:57:21.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:21.305+0000 7f86bd62d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:21.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:21.314+0000 7f86bd62d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:21.482 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 77 16 10545 10529 1.06819 1.13281 0.0784751 0.11693 2024-04-23T01:57:22.482 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 78 16 10665 10649 1.06651 0.9375 0.278363 0.117062 2024-04-23T01:57:23.482 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 79 16 10792 10776 1.06557 0.992188 0.0826991 0.117207 2024-04-23T01:57:23.532 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:57:23.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:23.664+0000 7ff3d2aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:23.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:23.671+0000 7ff3d2aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:24.482 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:57:24.481436+0000 min lat: 0.014953 max lat: 0.38169 avg lat: 0.117447 2024-04-23T01:57:24.482 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:57:24.482 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 80 16 10905 10889 1.06329 0.882812 0.0624364 0.117447 2024-04-23T01:57:25.482 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 81 16 11027 11011 1.06192 0.953125 0.0860184 0.117554 2024-04-23T01:57:25.657 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:57:25.658 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:57:25.658 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:57:25.658 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:57:25.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:25.774+0000 7f0ddee6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:25.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:25.784+0000 7f0ddee6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:25.829 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:57:25.829 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:57:25.829 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:57:25.829 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:57:25.829 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:57:25.829 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:57:25.829 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:57:25.829 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:57:25.830 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:57:25.830 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:57:25.830 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:57:25.830 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:57:25.832 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:57:25.832 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:57:25.832 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:57:25.915 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:57:25.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:25.957+0000 7f0926c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:25.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:25.975+0000 7f0926c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:26.008 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:57:26.008 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:57:26.008 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:57:26.008 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:57:26.008 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:57:26.008 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:57:26.008 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:57:26.008 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:57:26.009 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:57:26.009 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:57:26.009 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:57:26.009 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:57:26.013 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:57:26.013 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:57:26.013 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:57:26.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:26.051+0000 7f5458e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:26.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:26.058+0000 7f5458e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:26.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:26.142+0000 7f9b01224640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:26.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:26.153+0000 7f9b01224640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:26.186 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:57:26.186 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:57:26.187 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:57:26.187 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:57:26.187 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:57:26.187 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:57:26.187 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:57:26.187 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:57:26.187 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:57:26.187 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:57:26.187 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:57:26.187 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:57:26.189 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:57:26.190 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:57:26.190 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:57:26.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:26.383+0000 7f1db799d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:26.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:26.396+0000 7f1db799d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:26.420 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:57:26.421 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:57:26.421 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:57:26.421 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:57:26.421 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:57:26.421 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:57:26.421 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:57:26.421 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:57:26.421 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:57:26.421 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:57:26.421 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:57:26.421 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:57:26.424 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:57:26.424 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:57:26.482 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 82 16 11140 11124 1.05974 0.882812 0.123197 0.11783 2024-04-23T01:57:27.482 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 83 16 11251 11235 1.05742 0.867188 0.369638 0.11806 2024-04-23T01:57:28.212 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:57:28.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:28.352+0000 7fe66f217640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:28.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:28.360+0000 7fe66f217640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:28.482 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 84 16 11384 11368 1.0572 1.03906 0.0653278 0.118121 2024-04-23T01:57:29.482 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 85 16 11508 11492 1.05616 0.96875 0.0788291 0.118255 2024-04-23T01:57:30.482 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 86 16 11631 11615 1.05505 0.960938 0.0865752 0.118384 2024-04-23T01:57:30.633 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:57:30.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:30.764+0000 7f52874ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:30.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:30.772+0000 7f52874ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:31.424 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:57:31.425 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:57:31.425 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:57:31.425 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:57:31.482 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 87 16 11741 11725 1.0528 0.859375 0.146923 0.118612 2024-04-23T01:57:31.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:31.550+0000 7fd118e17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:31.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:31.562+0000 7fd118e17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:31.582 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:57:31.582 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:57:31.582 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:57:31.582 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:57:31.582 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:57:31.582 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:57:31.582 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:57:31.582 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:57:31.583 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:57:31.583 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:57:31.583 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:57:31.583 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:57:31.586 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:57:31.586 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:57:31.586 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:57:31.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:31.706+0000 7fbb3d99d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:31.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:31.718+0000 7fbb3d99d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:31.784 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:57:31.784 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:57:31.784 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:57:31.784 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:57:31.784 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:57:31.784 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:57:31.784 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:57:31.785 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:57:31.785 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:57:31.785 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:57:31.785 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:57:31.785 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:57:31.786 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:57:31.787 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:57:31.787 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:57:31.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:31.911+0000 7ff96235d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:31.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:31.922+0000 7ff96235d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:31.947 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:57:31.947 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:57:31.947 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:57:31.947 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:57:31.947 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:57:31.947 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:57:31.947 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:57:31.947 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:57:31.947 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:57:31.948 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:57:31.948 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:57:31.948 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:57:31.950 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:57:31.951 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:57:31.951 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:57:32.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:32.085+0000 7f584a39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:32.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:32.094+0000 7f584a39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:32.198 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:57:32.198 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:57:32.198 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:57:32.199 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:57:32.199 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:57:32.199 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:57:32.199 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:57:32.199 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:57:32.199 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:57:32.199 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:57:32.199 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:57:32.199 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:57:32.201 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:57:32.201 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:57:32.482 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 88 16 11858 11842 1.05122 0.914062 0.068767 0.118782 2024-04-23T01:57:33.071 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:57:33.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:33.186+0000 7fc5c926d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:33.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:33.193+0000 7fc5c926d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:33.482 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 89 16 11985 11969 1.05056 0.992188 0.197536 0.118862 2024-04-23T01:57:34.483 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 90 16 12115 12099 1.05017 1.01562 0.0939723 0.118887 2024-04-23T01:57:35.362 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:57:35.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:35.465+0000 7fedb866d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:35.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:35.471+0000 7fedb866d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:35.483 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Total time run: 90.1379 2024-04-23T01:57:35.483 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Total writes made: 12115 2024-04-23T01:57:35.483 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Write size: 8192 2024-04-23T01:57:35.483 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Object size: 65536 2024-04-23T01:57:35.483 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Bandwidth (MB/sec): 1.05004 2024-04-23T01:57:35.483 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Stddev Bandwidth: 0.103645 2024-04-23T01:57:35.483 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Max bandwidth (MB/sec): 1.36719 2024-04-23T01:57:35.483 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Min bandwidth (MB/sec): 0.804688 2024-04-23T01:57:35.483 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Average IOPS: 134 2024-04-23T01:57:35.483 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Stddev IOPS: 13.2665 2024-04-23T01:57:35.484 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Max IOPS: 175 2024-04-23T01:57:35.484 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Min IOPS: 103 2024-04-23T01:57:35.484 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Average Latency(s): 0.118946 2024-04-23T01:57:35.484 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Stddev Latency(s): 0.0596955 2024-04-23T01:57:35.484 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Max latency(s): 0.38169 2024-04-23T01:57:35.484 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Min latency(s): 0.014953 2024-04-23T01:57:35.510 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Cleaning up (deleting benchmark objects) 2024-04-23T01:57:37.203 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:57:37.203 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:57:37.203 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:57:37.203 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:57:37.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:37.330+0000 7f9d7199d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:37.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:37.342+0000 7f9d7199d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:37.449 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:57:37.449 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:57:37.449 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:57:37.449 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:57:37.450 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:57:37.450 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:57:37.450 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:57:37.450 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:57:37.450 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:57:37.450 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:57:37.450 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:57:37.450 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:57:37.453 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:57:37.453 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:57:37.454 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:57:37.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:37.572+0000 7f61e199d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:37.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:37.584+0000 7f61e199d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:37.620 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:57:37.620 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:57:37.620 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:57:37.621 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:57:37.621 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:57:37.621 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:57:37.621 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:57:37.621 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:57:37.621 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:57:37.621 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:57:37.621 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:57:37.621 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:57:37.623 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:57:37.623 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:57:37.624 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:57:37.739 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:57:37.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:37.739+0000 7f8efab5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:37.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:37.750+0000 7f8efab5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:37.778 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:57:37.778 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:57:37.778 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:57:37.778 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:57:37.778 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:57:37.778 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:57:37.778 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:57:37.778 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:57:37.778 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:57:37.778 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:57:37.778 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:57:37.779 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:57:37.781 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:57:37.781 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:57:37.781 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:57:37.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:37.865+0000 7f836555d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:37.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:37.876+0000 7f836555d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:37.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:37.960+0000 7f9947aed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:37.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:37.970+0000 7f9947aed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:38.006 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:57:38.006 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:57:38.006 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:57:38.007 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:57:38.007 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:57:38.007 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:57:38.007 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:57:38.007 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:57:38.007 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:57:38.007 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:57:38.007 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:57:38.007 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:57:38.011 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:57:38.011 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:57:40.302 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:57:40.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:40.436+0000 7f3e53d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:40.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:40.443+0000 7f3e53d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:42.774 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:57:42.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:42.909+0000 7fd05046d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:42.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:42.915+0000 7fd05046d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:43.012 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:57:43.012 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:57:43.012 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:57:43.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:43.140+0000 7fa7752ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:43.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:43.153+0000 7fa7752ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:43.497 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:57:43.498 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:57:43.516 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":451,"stamp":"2024-04-23T01:57:41.695679+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":87777848,"num_objects":1334,"num_object_clones":0,"num_object_copies":4002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":12437,"num_write_kb":98815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12381,"ondisk_log_size":12381,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":593108,"kb_used_data":135956,"kb_used_omap":0,"kb_used_meta":457088,"kb_avail":376894252,"statfs":{"total":386547056640,"available":385939714048,"internally_reserved":0,"allocated":139218944,"data_stored":275350480,"data_compressed":14163735,"data_compressed_allocated":136409088,"data_compressed_original":274292736,"omap_allocated":0,"internal_metadata":468058112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4227048,"num_objects":-64,"num_object_clones":0,"num_object_copies":-192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1141,"num_write_kb":7649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002663"},"pg_stats":[{"pgid":"9.7","version":"80'881","reported_seq":936,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T01:57:36.414399+0000","last_change":"2024-04-23T01:56:03.683930+0000","last_active":"2024-04-23T01:57:36.414399+0000","last_peered":"2024-04-23T01:57:36.414399+0000","last_clean":"2024-04-23T01:57:36.414399+0000","last_became_active":"2024-04-23T01:56:03.675554+0000","last_became_peered":"2024-04-23T01:56:03.675554+0000","last_unstale":"2024-04-23T01:57:36.414399+0000","last_undegraded":"2024-04-23T01:57:36.414399+0000","last_fullsized":"2024-04-23T01:57:36.414399+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":881,"log_dups_size":0,"ondisk_log_size":881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":882,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"80'856","reported_seq":911,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T01:57:36.611142+0000","last_change":"2024-04-23T01:56:03.552647+0000","last_active":"2024-04-23T01:57:36.611142+0000","last_peered":"2024-04-23T01:57:36.611142+0000","last_clean":"2024-04-23T01:57:36.611142+0000","last_became_active":"2024-04-23T01:56:03.541685+0000","last_became_peered":"2024-04-23T01:56:03.541685+0000","last_unstale":"2024-04-23T01:57:36.611142+0000","last_undegraded":"2024-04-23T01:57:36.611142+0000","last_fullsized":"2024-04-23T01:57:36.611142+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767168,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":857,"num_write_kb":6720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"80'763","reported_seq":816,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T01:57:36.293842+0000","last_change":"2024-04-23T01:56:03.515568+0000","last_active":"2024-04-23T01:57:36.293842+0000","last_peered":"2024-04-23T01:57:36.293842+0000","last_clean":"2024-04-23T01:57:36.293842+0000","last_became_active":"2024-04-23T01:56:03.510354+0000","last_became_peered":"2024-04-23T01:56:03.510354+0000","last_unstale":"2024-04-23T01:57:36.293842+0000","last_undegraded":"2024-04-23T01:57:36.293842+0000","last_fullsized":"2024-04-23T01:57:36.293842+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":763,"log_dups_size":0,"ondisk_log_size":763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":764,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.4","version":"80'813","reported_seq":866,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T01:57:36.712605+0000","last_change":"2024-04-23T01:56:03.647359+0000","last_active":"2024-04-23T01:57:36.712605+0000","last_peered":"2024-04-23T01:57:36.712605+0000","last_clean":"2024-04-23T01:57:36.712605+0000","last_became_active":"2024-04-23T01:56:03.642592+0000","last_became_peered":"2024-04-23T01:56:03.642592+0000","last_unstale":"2024-04-23T01:57:36.712605+0000","last_undegraded":"2024-04-23T01:57:36.712605+0000","last_fullsized":"2024-04-23T01:57:36.712605+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":813,"log_dups_size":0,"ondisk_log_size":813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5636096,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":814,"num_write_kb":6400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"80'786","reported_seq":841,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T01:57:36.573951+0000","last_change":"2024-04-23T01:56:03.566150+0000","last_active":"2024-04-23T01:57:36.573951+0000","last_peered":"2024-04-23T01:57:36.573951+0000","last_clean":"2024-04-23T01:57:36.573951+0000","last_became_active":"2024-04-23T01:56:03.553086+0000","last_became_peered":"2024-04-23T01:56:03.553086+0000","last_unstale":"2024-04-23T01:57:36.573951+0000","last_undegraded":"2024-04-23T01:57:36.573951+0000","last_fullsized":"2024-04-23T01:57:36.573951+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5636096,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":787,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"80'648","reported_seq":703,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T01:57:36.407102+0000","last_change":"2024-04-23T01:56:03.605892+0000","last_active":"2024-04-23T01:57:36.407102+0000","last_peered":"2024-04-23T01:57:36.407102+0000","last_clean":"2024-04-23T01:57:36.407102+0000","last_became_active":"2024-04-23T01:56:03.601950+0000","last_became_peered":"2024-04-23T01:56:03.601950+0000","last_unstale":"2024-04-23T01:57:36.407102+0000","last_undegraded":"2024-04-23T01:57:36.407102+0000","last_fullsized":"2024-04-23T01:57:36.407102+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":648,"log_dups_size":0,"ondisk_log_size":648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":649,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"80'560","reported_seq":615,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T01:57:36.566083+0000","last_change":"2024-04-23T01:56:03.618761+0000","last_active":"2024-04-23T01:57:36.566083+0000","last_peered":"2024-04-23T01:57:36.566083+0000","last_clean":"2024-04-23T01:57:36.566083+0000","last_became_active":"2024-04-23T01:56:03.602521+0000","last_became_peered":"2024-04-23T01:56:03.602521+0000","last_unstale":"2024-04-23T01:57:36.566083+0000","last_undegraded":"2024-04-23T01:57:36.566083+0000","last_fullsized":"2024-04-23T01:57:36.566083+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":561,"num_write_kb":4416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"80'731","reported_seq":784,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T01:57:36.405021+0000","last_change":"2024-04-23T01:56:03.544149+0000","last_active":"2024-04-23T01:57:36.405021+0000","last_peered":"2024-04-23T01:57:36.405021+0000","last_clean":"2024-04-23T01:57:36.405021+0000","last_became_active":"2024-04-23T01:56:03.539589+0000","last_became_peered":"2024-04-23T01:56:03.539589+0000","last_unstale":"2024-04-23T01:57:36.405021+0000","last_undegraded":"2024-04-23T01:57:36.405021+0000","last_fullsized":"2024-04-23T01:57:36.405021+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5111808,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":732,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"80'926","reported_seq":981,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T01:57:36.499170+0000","last_change":"2024-04-23T01:56:03.635647+0000","last_active":"2024-04-23T01:57:36.499170+0000","last_peered":"2024-04-23T01:57:36.499170+0000","last_clean":"2024-04-23T01:57:36.499170+0000","last_became_active":"2024-04-23T01:56:03.627392+0000","last_became_peered":"2024-04-23T01:56:03.627392+0000","last_unstale":"2024-04-23T01:57:36.499170+0000","last_undegraded":"2024-04-23T01:57:36.499170+0000","last_fullsized":"2024-04-23T01:57:36.499170+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6709248,"num_objects":103,"num_object_clones":0,"num_object_copies":309,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":927,"num_write_kb":7320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":321,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T01:57:19.257968+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:57:19.257968+0000","last_peered":"2024-04-23T01:57:19.257968+0000","last_clean":"2024-04-23T01:57:19.257968+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:57:19.257968+0000","last_undegraded":"2024-04-23T01:57:19.257968+0000","last_fullsized":"2024-04-23T01:57:19.257968+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":190,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T01:57:23.607280+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:57:23.607280+0000","last_peered":"2024-04-23T01:57:23.607280+0000","last_clean":"2024-04-23T01:57:23.607280+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:57:23.607280+0000","last_undegraded":"2024-04-23T01:57:23.607280+0000","last_fullsized":"2024-04-23T01:57:23.607280+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.b","version":"80'693","reported_seq":748,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T01:57:36.470680+0000","last_change":"2024-04-23T01:56:03.687235+0000","last_active":"2024-04-23T01:57:36.470680+0000","last_peered":"2024-04-23T01:57:36.470680+0000","last_clean":"2024-04-23T01:57:36.470680+0000","last_became_active":"2024-04-23T01:56:03.683119+0000","last_became_peered":"2024-04-23T01:56:03.683119+0000","last_unstale":"2024-04-23T01:57:36.470680+0000","last_undegraded":"2024-04-23T01:57:36.470680+0000","last_fullsized":"2024-04-23T01:57:36.470680+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":693,"log_dups_size":0,"ondisk_log_size":693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":694,"num_write_kb":5504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":182,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T01:57:23.627510+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:57:23.627510+0000","last_peered":"2024-04-23T01:57:23.627510+0000","last_clean":"2024-04-23T01:57:23.627510+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:57:23.627510+0000","last_undegraded":"2024-04-23T01:57:23.627510+0000","last_fullsized":"2024-04-23T01:57:23.627510+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"80'698","reported_seq":751,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T01:57:36.318290+0000","last_change":"2024-04-23T01:56:03.560440+0000","last_active":"2024-04-23T01:57:36.318290+0000","last_peered":"2024-04-23T01:57:36.318290+0000","last_clean":"2024-04-23T01:57:36.318290+0000","last_became_active":"2024-04-23T01:56:03.552840+0000","last_became_peered":"2024-04-23T01:56:03.552840+0000","last_unstale":"2024-04-23T01:57:36.318290+0000","last_undegraded":"2024-04-23T01:57:36.318290+0000","last_fullsized":"2024-04-23T01:57:36.318290+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":699,"num_write_kb":5504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":190,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T01:57:23.598871+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:57:23.598871+0000","last_peered":"2024-04-23T01:57:23.598871+0000","last_clean":"2024-04-23T01:57:23.598871+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:57:23.598871+0000","last_undegraded":"2024-04-23T01:57:23.598871+0000","last_fullsized":"2024-04-23T01:57:23.598871+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"80'776","reported_seq":831,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T01:57:36.556139+0000","last_change":"2024-04-23T01:56:03.597772+0000","last_active":"2024-04-23T01:57:36.556139+0000","last_peered":"2024-04-23T01:57:36.556139+0000","last_clean":"2024-04-23T01:57:36.556139+0000","last_became_active":"2024-04-23T01:56:03.589730+0000","last_became_peered":"2024-04-23T01:56:03.589730+0000","last_unstale":"2024-04-23T01:57:36.556139+0000","last_undegraded":"2024-04-23T01:57:36.556139+0000","last_fullsized":"2024-04-23T01:57:36.556139+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":776,"log_dups_size":0,"ondisk_log_size":776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":777,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":182,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T01:57:23.636393+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:57:23.636393+0000","last_peered":"2024-04-23T01:57:23.636393+0000","last_clean":"2024-04-23T01:57:23.636393+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:57:23.636393+0000","last_undegraded":"2024-04-23T01:57:23.636393+0000","last_fullsized":"2024-04-23T01:57:23.636393+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.f","version":"80'760","reported_seq":815,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T01:57:36.435494+0000","last_change":"2024-04-23T01:56:03.654933+0000","last_active":"2024-04-23T01:57:36.435494+0000","last_peered":"2024-04-23T01:57:36.435494+0000","last_clean":"2024-04-23T01:57:36.435494+0000","last_became_active":"2024-04-23T01:56:03.641185+0000","last_became_peered":"2024-04-23T01:56:03.641185+0000","last_unstale":"2024-04-23T01:57:36.435494+0000","last_undegraded":"2024-04-23T01:57:36.435494+0000","last_fullsized":"2024-04-23T01:57:36.435494+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":760,"log_dups_size":0,"ondisk_log_size":760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5570560,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":761,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":186,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T01:57:23.556534+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:57:23.556534+0000","last_peered":"2024-04-23T01:57:23.556534+0000","last_clean":"2024-04-23T01:57:23.556534+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:57:23.556534+0000","last_undegraded":"2024-04-23T01:57:23.556534+0000","last_fullsized":"2024-04-23T01:57:23.556534+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.e","version":"80'779","reported_seq":834,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T01:57:36.622748+0000","last_change":"2024-04-23T01:56:03.625197+0000","last_active":"2024-04-23T01:57:36.622748+0000","last_peered":"2024-04-23T01:57:36.622748+0000","last_clean":"2024-04-23T01:57:36.622748+0000","last_became_active":"2024-04-23T01:56:03.611498+0000","last_became_peered":"2024-04-23T01:56:03.611498+0000","last_unstale":"2024-04-23T01:57:36.622748+0000","last_undegraded":"2024-04-23T01:57:36.622748+0000","last_fullsized":"2024-04-23T01:57:36.622748+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":779,"log_dups_size":0,"ondisk_log_size":779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505024,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":780,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":188,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T01:57:19.255584+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:57:19.255584+0000","last_peered":"2024-04-23T01:57:19.255584+0000","last_clean":"2024-04-23T01:57:19.255584+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:57:19.255584+0000","last_undegraded":"2024-04-23T01:57:19.255584+0000","last_fullsized":"2024-04-23T01:57:19.255584+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"80'891","reported_seq":947,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T01:57:36.697553+0000","last_change":"2024-04-23T01:56:03.612020+0000","last_active":"2024-04-23T01:57:36.697553+0000","last_peered":"2024-04-23T01:57:36.697553+0000","last_clean":"2024-04-23T01:57:36.697553+0000","last_became_active":"2024-04-23T01:56:03.605401+0000","last_became_peered":"2024-04-23T01:56:03.605401+0000","last_unstale":"2024-04-23T01:57:36.697553+0000","last_undegraded":"2024-04-23T01:57:36.697553+0000","last_fullsized":"2024-04-23T01:57:36.697553+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6488088,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":892,"num_write_kb":7041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":186,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T01:57:23.553974+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:57:23.553974+0000","last_peered":"2024-04-23T01:57:23.553974+0000","last_clean":"2024-04-23T01:57:23.553974+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:57:23.553974+0000","last_undegraded":"2024-04-23T01:57:23.553974+0000","last_fullsized":"2024-04-23T01:57:23.553974+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.c","version":"80'723","reported_seq":778,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T01:57:36.641713+0000","last_change":"2024-04-23T01:56:03.574136+0000","last_active":"2024-04-23T01:57:36.641713+0000","last_peered":"2024-04-23T01:57:36.641713+0000","last_clean":"2024-04-23T01:57:36.641713+0000","last_became_active":"2024-04-23T01:56:03.566135+0000","last_became_peered":"2024-04-23T01:56:03.566135+0000","last_unstale":"2024-04-23T01:57:36.641713+0000","last_undegraded":"2024-04-23T01:57:36.641713+0000","last_fullsized":"2024-04-23T01:57:36.641713+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:56:02.366217+0000","last_clean_scrub_stamp":"2024-04-23T01:56:02.366217+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5046272,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":724,"num_write_kb":5696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":190,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-23T01:57:23.593980+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:57:23.593980+0000","last_peered":"2024-04-23T01:57:23.593980+0000","last_clean":"2024-04-23T01:57:23.593980+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:57:23.593980+0000","last_undegraded":"2024-04-23T01:57:23.593980+0000","last_fullsized":"2024-04-23T01:57:23.593980+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":87187480,"num_objects":1332,"num_object_clones":0,"num_object_copies":3996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12300,"num_write_kb":96921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":130596864,"data_stored":261169224,"data_compressed":13485660,"data_compressed_allocated":130584576,"data_compressed_original":261169152,"omap_allocated":0,"internal_metadata":0},"log_size":12284,"ondisk_log_size":12284,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706413,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":157456,"kb_used_data":36232,"kb_used_omap":0,"kb_used_meta":121216,"kb_avail":94214384,"statfs":{"total":96636764160,"available":96475529216,"internally_reserved":0,"allocated":37101568,"data_stored":73546114,"data_compressed":3784078,"data_compressed_allocated":36392960,"data_compressed_original":73277440,"omap_allocated":0,"internal_metadata":124125184},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706413,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":131460,"kb_used_data":28284,"kb_used_omap":0,"kb_used_meta":103168,"kb_avail":94240380,"statfs":{"total":96636764160,"available":96502149120,"internally_reserved":0,"allocated":28962816,"data_stored":57268610,"data_compressed":2943576,"data_compressed_allocated":28254208,"data_compressed_original":56999936,"omap_allocated":0,"internal_metadata":105644032},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706413,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":130064,"kb_used_data":26856,"kb_used_omap":0,"kb_used_meta":103168,"kb_avail":94241776,"statfs":{"total":96636764160,"available":96503578624,"internally_reserved":0,"allocated":27500544,"data_stored":53884746,"data_compressed":2769380,"data_compressed_allocated":26816512,"data_compressed_original":53633024,"omap_allocated":0,"internal_metadata":105644032},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706412,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":174128,"kb_used_data":44584,"kb_used_omap":0,"kb_used_meta":129536,"kb_avail":94197712,"statfs":{"total":96636764160,"available":96458457088,"internally_reserved":0,"allocated":45654016,"data_stored":90651010,"data_compressed":4666701,"data_compressed_allocated":44945408,"data_compressed_original":90382336,"omap_allocated":0,"internal_metadata":132644864},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40845312,"data_stored":81682456,"data_compressed":4217732,"data_compressed_allocated":40841216,"data_compressed_original":81682432,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26652672,"data_stored":53305344,"data_compressed":2752460,"data_compressed_allocated":26652672,"data_compressed_original":53305344,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27234304,"data_stored":54460440,"data_compressed":2812103,"data_compressed_allocated":27230208,"data_compressed_original":54460416,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":35864576,"data_stored":71720984,"data_compressed":3703365,"data_compressed_allocated":35860480,"data_compressed_original":71720960,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:57:45.147 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:57:45.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:45.285+0000 7f9a8ad9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:45.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:45.293+0000 7f9a8ad9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:46.055 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Removed 1515 objects 2024-04-23T01:57:46.055 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Clean up completed and total clean up time :10.5446 2024-04-23T01:57:47.546 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:57:47.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:47.669+0000 7f294731d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:47.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:47.675+0000 7f294731d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:48.135 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_6 2024-04-23T01:57:48.135 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_6 unique_pool_6 --yes-i-really-really-mean-it 2024-04-23T01:57:48.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:48.237+0000 7fe8f779d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:48.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:48.249+0000 7fe8f779d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:48.519 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:57:48.519 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-23T01:57:48.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:48.621+0000 7f35df79d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:48.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:48.630+0000 7f35df79d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:49.389 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'unique_pool_6' removed 2024-04-23T01:57:49.390 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.0 to 1 (10000) 2024-04-23T01:57:49.408 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-04-23T01:57:49.409 INFO:tasks.radosbench:Beginning radosbench... 2024-04-23T01:57:49.409 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_7 2024-04-23T01:57:49.409 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_7 16 2024-04-23T01:57:49.411 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-23T01:57:49.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:49.554+0000 7faeaab1d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:49.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:49.554+0000 7f30c14ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:49.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:49.565+0000 7faeaab1d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:49.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:49.566+0000 7f30c14ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:49.776 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:57:49.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:49.877+0000 7f17a7a0f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:49.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:49.883+0000 7f17a7a0f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:50.395 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.1 to 1 (10000) 2024-04-23T01:57:50.396 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'unique_pool_7' created 2024-04-23T01:57:50.413 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_7 rados --yes-i-really-mean-it || true 2024-04-23T01:57:50.421 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-23T01:57:50.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:50.569+0000 7f3e7086d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:50.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:50.590+0000 7f3e7086d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:50.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:50.625+0000 7fccd742d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:50.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:50.640+0000 7fccd742d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:51.399 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.2 to 1 (10000) 2024-04-23T01:57:51.399 INFO:teuthology.orchestra.run.smithi090.stderr:enabled application 'rados' on pool 'unique_pool_7' 2024-04-23T01:57:51.417 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-23T01:57:51.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:51.564+0000 7f391d39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:51.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:51.580+0000 7f391d39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:51.977 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:57:52.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:52.079+0000 7f6fc5f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:52.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:52.084+0000 7f6fc5f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:52.402 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.3 to 1 (10000) 2024-04-23T01:57:52.417 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-23T01:57:52.417 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:57:52.417 DEBUG:teuthology.orchestra.run.smithi090:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 8192 --object-size 65536 -p unique_pool_7 bench 90 write' 2024-04-23T01:57:52.420 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-04-23T01:57:52.420 INFO:teuthology.orchestra.run:waiting for 3000 2024-04-23T01:57:52.480 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:hints = 1 2024-04-23T01:57:52.481 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Maintaining 16 concurrent writes of 8192 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-04-23T01:57:52.481 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Object prefix: benchmark_data_smithi090_60427 2024-04-23T01:57:52.481 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:57:52.481 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 0 0 0 0 0 0 - 0 2024-04-23T01:57:52.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:52.530+0000 7f4532754640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:52.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:52.538+0000 7f4532754640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:52.911 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:57:52.912 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:57:52.933 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2024-04-23T01:57:52.402642+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":464228,"kb_used_data":2948,"kb_used_omap":0,"kb_used_meta":461248,"kb_avail":377023132,"statfs":{"total":386547056640,"available":386071687168,"internally_reserved":0,"allocated":3018752,"data_stored":2812526,"data_compressed":89859,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":472317952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-13632,"num_write_kb":-96921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.705634"},"pg_stats":[{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.582523+0000","last_change":"2024-04-23T01:57:51.583793+0000","last_active":"2024-04-23T01:57:51.582523+0000","last_peered":"2024-04-23T01:57:51.582523+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.574869+0000","last_became_peered":"2024-04-23T01:57:51.574869+0000","last_unstale":"2024-04-23T01:57:51.582523+0000","last_undegraded":"2024-04-23T01:57:51.582523+0000","last_fullsized":"2024-04-23T01:57:51.582523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.678020+0000","last_change":"2024-04-23T01:57:51.679265+0000","last_active":"2024-04-23T01:57:51.678020+0000","last_peered":"2024-04-23T01:57:51.678020+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.662859+0000","last_became_peered":"2024-04-23T01:57:51.662859+0000","last_unstale":"2024-04-23T01:57:51.678020+0000","last_undegraded":"2024-04-23T01:57:51.678020+0000","last_fullsized":"2024-04-23T01:57:51.678020+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.592258+0000","last_change":"2024-04-23T01:57:51.593118+0000","last_active":"2024-04-23T01:57:51.592258+0000","last_peered":"2024-04-23T01:57:51.592258+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.580553+0000","last_became_peered":"2024-04-23T01:57:51.580553+0000","last_unstale":"2024-04-23T01:57:51.592258+0000","last_undegraded":"2024-04-23T01:57:51.592258+0000","last_fullsized":"2024-04-23T01:57:51.592258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.575835+0000","last_change":"2024-04-23T01:57:51.577069+0000","last_active":"2024-04-23T01:57:51.575835+0000","last_peered":"2024-04-23T01:57:51.575835+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.568227+0000","last_became_peered":"2024-04-23T01:57:51.568227+0000","last_unstale":"2024-04-23T01:57:51.575835+0000","last_undegraded":"2024-04-23T01:57:51.575835+0000","last_fullsized":"2024-04-23T01:57:51.575835+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":4,"reported_epoch":83,"state":"creating+activating","last_fresh":"2024-04-23T01:57:51.460986+0000","last_change":"2024-04-23T01:57:51.460986+0000","last_active":"2024-04-23T01:57:50.388836+0000","last_peered":"2024-04-23T01:57:50.388836+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:57:51.460986+0000","last_undegraded":"2024-04-23T01:57:51.460986+0000","last_fullsized":"2024-04-23T01:57:51.460986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":["0","2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.628393+0000","last_change":"2024-04-23T01:57:51.630095+0000","last_active":"2024-04-23T01:57:51.628393+0000","last_peered":"2024-04-23T01:57:51.628393+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.623783+0000","last_became_peered":"2024-04-23T01:57:51.623783+0000","last_unstale":"2024-04-23T01:57:51.628393+0000","last_undegraded":"2024-04-23T01:57:51.628393+0000","last_fullsized":"2024-04-23T01:57:51.628393+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.603232+0000","last_change":"2024-04-23T01:57:51.605374+0000","last_active":"2024-04-23T01:57:51.603232+0000","last_peered":"2024-04-23T01:57:51.603232+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.597053+0000","last_became_peered":"2024-04-23T01:57:51.597053+0000","last_unstale":"2024-04-23T01:57:51.603232+0000","last_undegraded":"2024-04-23T01:57:51.603232+0000","last_fullsized":"2024-04-23T01:57:51.603232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.640703+0000","last_change":"2024-04-23T01:57:51.641687+0000","last_active":"2024-04-23T01:57:51.640703+0000","last_peered":"2024-04-23T01:57:51.640703+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.623973+0000","last_became_peered":"2024-04-23T01:57:51.623973+0000","last_unstale":"2024-04-23T01:57:51.640703+0000","last_undegraded":"2024-04-23T01:57:51.640703+0000","last_fullsized":"2024-04-23T01:57:51.640703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.668799+0000","last_change":"2024-04-23T01:57:51.669918+0000","last_active":"2024-04-23T01:57:51.668799+0000","last_peered":"2024-04-23T01:57:51.668799+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.652610+0000","last_became_peered":"2024-04-23T01:57:51.652610+0000","last_unstale":"2024-04-23T01:57:51.668799+0000","last_undegraded":"2024-04-23T01:57:51.668799+0000","last_fullsized":"2024-04-23T01:57:51.668799+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":328,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.410996+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:57:51.410996+0000","last_peered":"2024-04-23T01:57:51.410996+0000","last_clean":"2024-04-23T01:57:51.410996+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:57:51.410996+0000","last_undegraded":"2024-04-23T01:57:51.410996+0000","last_fullsized":"2024-04-23T01:57:51.410996+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":197,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.435401+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:57:51.435401+0000","last_peered":"2024-04-23T01:57:51.435401+0000","last_clean":"2024-04-23T01:57:51.435401+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:57:51.435401+0000","last_undegraded":"2024-04-23T01:57:51.435401+0000","last_fullsized":"2024-04-23T01:57:51.435401+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.586325+0000","last_change":"2024-04-23T01:57:51.587471+0000","last_active":"2024-04-23T01:57:51.586325+0000","last_peered":"2024-04-23T01:57:51.586325+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.575352+0000","last_became_peered":"2024-04-23T01:57:51.575352+0000","last_unstale":"2024-04-23T01:57:51.586325+0000","last_undegraded":"2024-04-23T01:57:51.586325+0000","last_fullsized":"2024-04-23T01:57:51.586325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":189,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.475424+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:57:51.475424+0000","last_peered":"2024-04-23T01:57:51.475424+0000","last_clean":"2024-04-23T01:57:51.475424+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:57:51.475424+0000","last_undegraded":"2024-04-23T01:57:51.475424+0000","last_fullsized":"2024-04-23T01:57:51.475424+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.594969+0000","last_change":"2024-04-23T01:57:51.596134+0000","last_active":"2024-04-23T01:57:51.594969+0000","last_peered":"2024-04-23T01:57:51.594969+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.589605+0000","last_became_peered":"2024-04-23T01:57:51.589605+0000","last_unstale":"2024-04-23T01:57:51.594969+0000","last_undegraded":"2024-04-23T01:57:51.594969+0000","last_fullsized":"2024-04-23T01:57:51.594969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":197,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.442198+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:57:51.442198+0000","last_peered":"2024-04-23T01:57:51.442198+0000","last_clean":"2024-04-23T01:57:51.442198+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:57:51.442198+0000","last_undegraded":"2024-04-23T01:57:51.442198+0000","last_fullsized":"2024-04-23T01:57:51.442198+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.660500+0000","last_change":"2024-04-23T01:57:51.662237+0000","last_active":"2024-04-23T01:57:51.660500+0000","last_peered":"2024-04-23T01:57:51.660500+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.647583+0000","last_became_peered":"2024-04-23T01:57:51.647583+0000","last_unstale":"2024-04-23T01:57:51.660500+0000","last_undegraded":"2024-04-23T01:57:51.660500+0000","last_fullsized":"2024-04-23T01:57:51.660500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":189,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.480887+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:57:51.480887+0000","last_peered":"2024-04-23T01:57:51.480887+0000","last_clean":"2024-04-23T01:57:51.480887+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:57:51.480887+0000","last_undegraded":"2024-04-23T01:57:51.480887+0000","last_fullsized":"2024-04-23T01:57:51.480887+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":7,"reported_epoch":83,"state":"active","last_fresh":"2024-04-23T01:57:51.670788+0000","last_change":"2024-04-23T01:57:51.670788+0000","last_active":"2024-04-23T01:57:51.670788+0000","last_peered":"2024-04-23T01:57:51.670788+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.670788+0000","last_became_peered":"2024-04-23T01:57:51.670788+0000","last_unstale":"2024-04-23T01:57:51.670788+0000","last_undegraded":"2024-04-23T01:57:51.670788+0000","last_fullsized":"2024-04-23T01:57:51.670788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":["0","2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":195,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.427075+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:57:51.427075+0000","last_peered":"2024-04-23T01:57:51.427075+0000","last_clean":"2024-04-23T01:57:51.427075+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:57:51.427075+0000","last_undegraded":"2024-04-23T01:57:51.427075+0000","last_fullsized":"2024-04-23T01:57:51.427075+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.630684+0000","last_change":"2024-04-23T01:57:51.632423+0000","last_active":"2024-04-23T01:57:51.630684+0000","last_peered":"2024-04-23T01:57:51.630684+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.624354+0000","last_became_peered":"2024-04-23T01:57:51.624354+0000","last_unstale":"2024-04-23T01:57:51.630684+0000","last_undegraded":"2024-04-23T01:57:51.630684+0000","last_fullsized":"2024-04-23T01:57:51.630684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":195,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.459889+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:57:51.459889+0000","last_peered":"2024-04-23T01:57:51.459889+0000","last_clean":"2024-04-23T01:57:51.459889+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:57:51.459889+0000","last_undegraded":"2024-04-23T01:57:51.459889+0000","last_fullsized":"2024-04-23T01:57:51.459889+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":4,"reported_epoch":83,"state":"creating+activating","last_fresh":"2024-04-23T01:57:51.436485+0000","last_change":"2024-04-23T01:57:51.436485+0000","last_active":"2024-04-23T01:57:50.388836+0000","last_peered":"2024-04-23T01:57:50.388836+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:57:51.436485+0000","last_undegraded":"2024-04-23T01:57:51.436485+0000","last_fullsized":"2024-04-23T01:57:51.436485+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":["0","1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":195,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.431160+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:57:51.431160+0000","last_peered":"2024-04-23T01:57:51.431160+0000","last_clean":"2024-04-23T01:57:51.431160+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:57:51.431160+0000","last_undegraded":"2024-04-23T01:57:51.431160+0000","last_fullsized":"2024-04-23T01:57:51.431160+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.655093+0000","last_change":"2024-04-23T01:57:51.656207+0000","last_active":"2024-04-23T01:57:51.655093+0000","last_peered":"2024-04-23T01:57:51.655093+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.638689+0000","last_became_peered":"2024-04-23T01:57:51.638689+0000","last_unstale":"2024-04-23T01:57:51.655093+0000","last_undegraded":"2024-04-23T01:57:51.655093+0000","last_fullsized":"2024-04-23T01:57:51.655093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":197,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.440088+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:57:51.440088+0000","last_peered":"2024-04-23T01:57:51.440088+0000","last_clean":"2024-04-23T01:57:51.440088+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:57:51.440088+0000","last_undegraded":"2024-04-23T01:57:51.440088+0000","last_fullsized":"2024-04-23T01:57:51.440088+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706428,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":126140,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":125376,"kb_avail":94245700,"statfs":{"total":96636764160,"available":96507596800,"internally_reserved":0,"allocated":774144,"data_stored":852532,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":128385024},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706428,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":108092,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":107328,"kb_avail":94263748,"statfs":{"total":96636764160,"available":96526077952,"internally_reserved":0,"allocated":774144,"data_stored":852532,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":109903872},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706428,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":108024,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":107328,"kb_avail":94263816,"statfs":{"total":96636764160,"available":96526147584,"internally_reserved":0,"allocated":704512,"data_stored":258547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":109903872},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706427,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":121972,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":121216,"kb_avail":94249868,"statfs":{"total":96636764160,"available":96511864832,"internally_reserved":0,"allocated":765952,"data_stored":848915,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":124125184},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:57:52.935 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:57:53.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:53.054+0000 7ffbc266d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:53.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:53.065+0000 7ffbc266d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:53.275 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:57:53.276 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:57:53.289 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2024-04-23T01:57:52.402642+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":464228,"kb_used_data":2948,"kb_used_omap":0,"kb_used_meta":461248,"kb_avail":377023132,"statfs":{"total":386547056640,"available":386071687168,"internally_reserved":0,"allocated":3018752,"data_stored":2812526,"data_compressed":89859,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":472317952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-13632,"num_write_kb":-96921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.705634"},"pg_stats":[{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.582523+0000","last_change":"2024-04-23T01:57:51.583793+0000","last_active":"2024-04-23T01:57:51.582523+0000","last_peered":"2024-04-23T01:57:51.582523+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.574869+0000","last_became_peered":"2024-04-23T01:57:51.574869+0000","last_unstale":"2024-04-23T01:57:51.582523+0000","last_undegraded":"2024-04-23T01:57:51.582523+0000","last_fullsized":"2024-04-23T01:57:51.582523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.678020+0000","last_change":"2024-04-23T01:57:51.679265+0000","last_active":"2024-04-23T01:57:51.678020+0000","last_peered":"2024-04-23T01:57:51.678020+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.662859+0000","last_became_peered":"2024-04-23T01:57:51.662859+0000","last_unstale":"2024-04-23T01:57:51.678020+0000","last_undegraded":"2024-04-23T01:57:51.678020+0000","last_fullsized":"2024-04-23T01:57:51.678020+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.592258+0000","last_change":"2024-04-23T01:57:51.593118+0000","last_active":"2024-04-23T01:57:51.592258+0000","last_peered":"2024-04-23T01:57:51.592258+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.580553+0000","last_became_peered":"2024-04-23T01:57:51.580553+0000","last_unstale":"2024-04-23T01:57:51.592258+0000","last_undegraded":"2024-04-23T01:57:51.592258+0000","last_fullsized":"2024-04-23T01:57:51.592258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.575835+0000","last_change":"2024-04-23T01:57:51.577069+0000","last_active":"2024-04-23T01:57:51.575835+0000","last_peered":"2024-04-23T01:57:51.575835+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.568227+0000","last_became_peered":"2024-04-23T01:57:51.568227+0000","last_unstale":"2024-04-23T01:57:51.575835+0000","last_undegraded":"2024-04-23T01:57:51.575835+0000","last_fullsized":"2024-04-23T01:57:51.575835+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":4,"reported_epoch":83,"state":"creating+activating","last_fresh":"2024-04-23T01:57:51.460986+0000","last_change":"2024-04-23T01:57:51.460986+0000","last_active":"2024-04-23T01:57:50.388836+0000","last_peered":"2024-04-23T01:57:50.388836+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:57:51.460986+0000","last_undegraded":"2024-04-23T01:57:51.460986+0000","last_fullsized":"2024-04-23T01:57:51.460986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":["0","2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.628393+0000","last_change":"2024-04-23T01:57:51.630095+0000","last_active":"2024-04-23T01:57:51.628393+0000","last_peered":"2024-04-23T01:57:51.628393+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.623783+0000","last_became_peered":"2024-04-23T01:57:51.623783+0000","last_unstale":"2024-04-23T01:57:51.628393+0000","last_undegraded":"2024-04-23T01:57:51.628393+0000","last_fullsized":"2024-04-23T01:57:51.628393+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.603232+0000","last_change":"2024-04-23T01:57:51.605374+0000","last_active":"2024-04-23T01:57:51.603232+0000","last_peered":"2024-04-23T01:57:51.603232+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.597053+0000","last_became_peered":"2024-04-23T01:57:51.597053+0000","last_unstale":"2024-04-23T01:57:51.603232+0000","last_undegraded":"2024-04-23T01:57:51.603232+0000","last_fullsized":"2024-04-23T01:57:51.603232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.640703+0000","last_change":"2024-04-23T01:57:51.641687+0000","last_active":"2024-04-23T01:57:51.640703+0000","last_peered":"2024-04-23T01:57:51.640703+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.623973+0000","last_became_peered":"2024-04-23T01:57:51.623973+0000","last_unstale":"2024-04-23T01:57:51.640703+0000","last_undegraded":"2024-04-23T01:57:51.640703+0000","last_fullsized":"2024-04-23T01:57:51.640703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.668799+0000","last_change":"2024-04-23T01:57:51.669918+0000","last_active":"2024-04-23T01:57:51.668799+0000","last_peered":"2024-04-23T01:57:51.668799+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.652610+0000","last_became_peered":"2024-04-23T01:57:51.652610+0000","last_unstale":"2024-04-23T01:57:51.668799+0000","last_undegraded":"2024-04-23T01:57:51.668799+0000","last_fullsized":"2024-04-23T01:57:51.668799+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":328,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.410996+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:57:51.410996+0000","last_peered":"2024-04-23T01:57:51.410996+0000","last_clean":"2024-04-23T01:57:51.410996+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:57:51.410996+0000","last_undegraded":"2024-04-23T01:57:51.410996+0000","last_fullsized":"2024-04-23T01:57:51.410996+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":197,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.435401+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:57:51.435401+0000","last_peered":"2024-04-23T01:57:51.435401+0000","last_clean":"2024-04-23T01:57:51.435401+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:57:51.435401+0000","last_undegraded":"2024-04-23T01:57:51.435401+0000","last_fullsized":"2024-04-23T01:57:51.435401+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.586325+0000","last_change":"2024-04-23T01:57:51.587471+0000","last_active":"2024-04-23T01:57:51.586325+0000","last_peered":"2024-04-23T01:57:51.586325+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.575352+0000","last_became_peered":"2024-04-23T01:57:51.575352+0000","last_unstale":"2024-04-23T01:57:51.586325+0000","last_undegraded":"2024-04-23T01:57:51.586325+0000","last_fullsized":"2024-04-23T01:57:51.586325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":189,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.475424+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:57:51.475424+0000","last_peered":"2024-04-23T01:57:51.475424+0000","last_clean":"2024-04-23T01:57:51.475424+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:57:51.475424+0000","last_undegraded":"2024-04-23T01:57:51.475424+0000","last_fullsized":"2024-04-23T01:57:51.475424+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.594969+0000","last_change":"2024-04-23T01:57:51.596134+0000","last_active":"2024-04-23T01:57:51.594969+0000","last_peered":"2024-04-23T01:57:51.594969+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.589605+0000","last_became_peered":"2024-04-23T01:57:51.589605+0000","last_unstale":"2024-04-23T01:57:51.594969+0000","last_undegraded":"2024-04-23T01:57:51.594969+0000","last_fullsized":"2024-04-23T01:57:51.594969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":197,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.442198+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:57:51.442198+0000","last_peered":"2024-04-23T01:57:51.442198+0000","last_clean":"2024-04-23T01:57:51.442198+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:57:51.442198+0000","last_undegraded":"2024-04-23T01:57:51.442198+0000","last_fullsized":"2024-04-23T01:57:51.442198+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.660500+0000","last_change":"2024-04-23T01:57:51.662237+0000","last_active":"2024-04-23T01:57:51.660500+0000","last_peered":"2024-04-23T01:57:51.660500+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.647583+0000","last_became_peered":"2024-04-23T01:57:51.647583+0000","last_unstale":"2024-04-23T01:57:51.660500+0000","last_undegraded":"2024-04-23T01:57:51.660500+0000","last_fullsized":"2024-04-23T01:57:51.660500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":189,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.480887+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:57:51.480887+0000","last_peered":"2024-04-23T01:57:51.480887+0000","last_clean":"2024-04-23T01:57:51.480887+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:57:51.480887+0000","last_undegraded":"2024-04-23T01:57:51.480887+0000","last_fullsized":"2024-04-23T01:57:51.480887+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":7,"reported_epoch":83,"state":"active","last_fresh":"2024-04-23T01:57:51.670788+0000","last_change":"2024-04-23T01:57:51.670788+0000","last_active":"2024-04-23T01:57:51.670788+0000","last_peered":"2024-04-23T01:57:51.670788+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.670788+0000","last_became_peered":"2024-04-23T01:57:51.670788+0000","last_unstale":"2024-04-23T01:57:51.670788+0000","last_undegraded":"2024-04-23T01:57:51.670788+0000","last_fullsized":"2024-04-23T01:57:51.670788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":["0","2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":195,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.427075+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:57:51.427075+0000","last_peered":"2024-04-23T01:57:51.427075+0000","last_clean":"2024-04-23T01:57:51.427075+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:57:51.427075+0000","last_undegraded":"2024-04-23T01:57:51.427075+0000","last_fullsized":"2024-04-23T01:57:51.427075+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.630684+0000","last_change":"2024-04-23T01:57:51.632423+0000","last_active":"2024-04-23T01:57:51.630684+0000","last_peered":"2024-04-23T01:57:51.630684+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.624354+0000","last_became_peered":"2024-04-23T01:57:51.624354+0000","last_unstale":"2024-04-23T01:57:51.630684+0000","last_undegraded":"2024-04-23T01:57:51.630684+0000","last_fullsized":"2024-04-23T01:57:51.630684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":195,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.459889+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:57:51.459889+0000","last_peered":"2024-04-23T01:57:51.459889+0000","last_clean":"2024-04-23T01:57:51.459889+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:57:51.459889+0000","last_undegraded":"2024-04-23T01:57:51.459889+0000","last_fullsized":"2024-04-23T01:57:51.459889+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":4,"reported_epoch":83,"state":"creating+activating","last_fresh":"2024-04-23T01:57:51.436485+0000","last_change":"2024-04-23T01:57:51.436485+0000","last_active":"2024-04-23T01:57:50.388836+0000","last_peered":"2024-04-23T01:57:50.388836+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:57:51.436485+0000","last_undegraded":"2024-04-23T01:57:51.436485+0000","last_fullsized":"2024-04-23T01:57:51.436485+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":["0","1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":195,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.431160+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:57:51.431160+0000","last_peered":"2024-04-23T01:57:51.431160+0000","last_clean":"2024-04-23T01:57:51.431160+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:57:51.431160+0000","last_undegraded":"2024-04-23T01:57:51.431160+0000","last_fullsized":"2024-04-23T01:57:51.431160+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.655093+0000","last_change":"2024-04-23T01:57:51.656207+0000","last_active":"2024-04-23T01:57:51.655093+0000","last_peered":"2024-04-23T01:57:51.655093+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.638689+0000","last_became_peered":"2024-04-23T01:57:51.638689+0000","last_unstale":"2024-04-23T01:57:51.655093+0000","last_undegraded":"2024-04-23T01:57:51.655093+0000","last_fullsized":"2024-04-23T01:57:51.655093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":197,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.440088+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:57:51.440088+0000","last_peered":"2024-04-23T01:57:51.440088+0000","last_clean":"2024-04-23T01:57:51.440088+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:57:51.440088+0000","last_undegraded":"2024-04-23T01:57:51.440088+0000","last_fullsized":"2024-04-23T01:57:51.440088+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706428,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":126140,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":125376,"kb_avail":94245700,"statfs":{"total":96636764160,"available":96507596800,"internally_reserved":0,"allocated":774144,"data_stored":852532,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":128385024},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706428,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":108092,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":107328,"kb_avail":94263748,"statfs":{"total":96636764160,"available":96526077952,"internally_reserved":0,"allocated":774144,"data_stored":852532,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":109903872},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706428,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":108024,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":107328,"kb_avail":94263816,"statfs":{"total":96636764160,"available":96526147584,"internally_reserved":0,"allocated":704512,"data_stored":258547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":109903872},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706427,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":121972,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":121216,"kb_avail":94249868,"statfs":{"total":96636764160,"available":96511864832,"internally_reserved":0,"allocated":765952,"data_stored":848915,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":124125184},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:57:53.291 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-23T01:57:53.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:53.406+0000 7f8038f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:53.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:53.415+0000 7f8038f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:53.480 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 1 16 119 103 0.80464 0.804688 0.0979747 0.141287 2024-04-23T01:57:53.757 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:57:53.757 INFO:teuthology.orchestra.run.smithi090.stdout:{"fsid":"0a731bfd-0fc4-4c1b-8e4b-f6dfbc059bcc","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":774,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":84,"num_osds":4,"num_up_osds":4,"osd_up_since":1713836706,"num_in_osds":4,"osd_in_since":1713836701,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":16},{"state_name":"active+clean","count":9}],"num_pgs":25,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":476172288,"bytes_avail":386070884352,"bytes_total":386547056640,"unknown_pgs_ratio":0.63999998569488525},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-04-23T01:46:05.559282+0000","services":{}},"progress_events":{}} 2024-04-23T01:57:53.774 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-04-23T01:57:53.775 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:57:53.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:53.897+0000 7ff5ffb1d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:53.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:53.907+0000 7ff5ffb1d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:54.128 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:57:54.128 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:57:54.144 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-04-23T01:57:53.698203+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":464228,"kb_used_data":2948,"kb_used_omap":0,"kb_used_meta":461248,"kb_avail":377023132,"statfs":{"total":386547056640,"available":386071687168,"internally_reserved":0,"allocated":3018752,"data_stored":2812526,"data_compressed":89859,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":472317952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.309214"},"pg_stats":[{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.582523+0000","last_change":"2024-04-23T01:57:51.583793+0000","last_active":"2024-04-23T01:57:51.582523+0000","last_peered":"2024-04-23T01:57:51.582523+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.574869+0000","last_became_peered":"2024-04-23T01:57:51.574869+0000","last_unstale":"2024-04-23T01:57:51.582523+0000","last_undegraded":"2024-04-23T01:57:51.582523+0000","last_fullsized":"2024-04-23T01:57:51.582523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.678020+0000","last_change":"2024-04-23T01:57:51.679265+0000","last_active":"2024-04-23T01:57:51.678020+0000","last_peered":"2024-04-23T01:57:51.678020+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.662859+0000","last_became_peered":"2024-04-23T01:57:51.662859+0000","last_unstale":"2024-04-23T01:57:51.678020+0000","last_undegraded":"2024-04-23T01:57:51.678020+0000","last_fullsized":"2024-04-23T01:57:51.678020+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.592258+0000","last_change":"2024-04-23T01:57:51.593118+0000","last_active":"2024-04-23T01:57:51.592258+0000","last_peered":"2024-04-23T01:57:51.592258+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.580553+0000","last_became_peered":"2024-04-23T01:57:51.580553+0000","last_unstale":"2024-04-23T01:57:51.592258+0000","last_undegraded":"2024-04-23T01:57:51.592258+0000","last_fullsized":"2024-04-23T01:57:51.592258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.575835+0000","last_change":"2024-04-23T01:57:51.577069+0000","last_active":"2024-04-23T01:57:51.575835+0000","last_peered":"2024-04-23T01:57:51.575835+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.568227+0000","last_became_peered":"2024-04-23T01:57:51.568227+0000","last_unstale":"2024-04-23T01:57:51.575835+0000","last_undegraded":"2024-04-23T01:57:51.575835+0000","last_fullsized":"2024-04-23T01:57:51.575835+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":4,"reported_epoch":83,"state":"creating+activating","last_fresh":"2024-04-23T01:57:51.460986+0000","last_change":"2024-04-23T01:57:51.460986+0000","last_active":"2024-04-23T01:57:50.388836+0000","last_peered":"2024-04-23T01:57:50.388836+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:57:51.460986+0000","last_undegraded":"2024-04-23T01:57:51.460986+0000","last_fullsized":"2024-04-23T01:57:51.460986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":["0","2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.628393+0000","last_change":"2024-04-23T01:57:51.630095+0000","last_active":"2024-04-23T01:57:51.628393+0000","last_peered":"2024-04-23T01:57:51.628393+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.623783+0000","last_became_peered":"2024-04-23T01:57:51.623783+0000","last_unstale":"2024-04-23T01:57:51.628393+0000","last_undegraded":"2024-04-23T01:57:51.628393+0000","last_fullsized":"2024-04-23T01:57:51.628393+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.603232+0000","last_change":"2024-04-23T01:57:51.605374+0000","last_active":"2024-04-23T01:57:51.603232+0000","last_peered":"2024-04-23T01:57:51.603232+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.597053+0000","last_became_peered":"2024-04-23T01:57:51.597053+0000","last_unstale":"2024-04-23T01:57:51.603232+0000","last_undegraded":"2024-04-23T01:57:51.603232+0000","last_fullsized":"2024-04-23T01:57:51.603232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.640703+0000","last_change":"2024-04-23T01:57:51.641687+0000","last_active":"2024-04-23T01:57:51.640703+0000","last_peered":"2024-04-23T01:57:51.640703+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.623973+0000","last_became_peered":"2024-04-23T01:57:51.623973+0000","last_unstale":"2024-04-23T01:57:51.640703+0000","last_undegraded":"2024-04-23T01:57:51.640703+0000","last_fullsized":"2024-04-23T01:57:51.640703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.668799+0000","last_change":"2024-04-23T01:57:51.669918+0000","last_active":"2024-04-23T01:57:51.668799+0000","last_peered":"2024-04-23T01:57:51.668799+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.652610+0000","last_became_peered":"2024-04-23T01:57:51.652610+0000","last_unstale":"2024-04-23T01:57:51.668799+0000","last_undegraded":"2024-04-23T01:57:51.668799+0000","last_fullsized":"2024-04-23T01:57:51.668799+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":328,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.410996+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:57:51.410996+0000","last_peered":"2024-04-23T01:57:51.410996+0000","last_clean":"2024-04-23T01:57:51.410996+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:57:51.410996+0000","last_undegraded":"2024-04-23T01:57:51.410996+0000","last_fullsized":"2024-04-23T01:57:51.410996+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":197,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.435401+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:57:51.435401+0000","last_peered":"2024-04-23T01:57:51.435401+0000","last_clean":"2024-04-23T01:57:51.435401+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:57:51.435401+0000","last_undegraded":"2024-04-23T01:57:51.435401+0000","last_fullsized":"2024-04-23T01:57:51.435401+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.586325+0000","last_change":"2024-04-23T01:57:51.587471+0000","last_active":"2024-04-23T01:57:51.586325+0000","last_peered":"2024-04-23T01:57:51.586325+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.575352+0000","last_became_peered":"2024-04-23T01:57:51.575352+0000","last_unstale":"2024-04-23T01:57:51.586325+0000","last_undegraded":"2024-04-23T01:57:51.586325+0000","last_fullsized":"2024-04-23T01:57:51.586325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":189,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.475424+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:57:51.475424+0000","last_peered":"2024-04-23T01:57:51.475424+0000","last_clean":"2024-04-23T01:57:51.475424+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:57:51.475424+0000","last_undegraded":"2024-04-23T01:57:51.475424+0000","last_fullsized":"2024-04-23T01:57:51.475424+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.594969+0000","last_change":"2024-04-23T01:57:51.596134+0000","last_active":"2024-04-23T01:57:51.594969+0000","last_peered":"2024-04-23T01:57:51.594969+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.589605+0000","last_became_peered":"2024-04-23T01:57:51.589605+0000","last_unstale":"2024-04-23T01:57:51.594969+0000","last_undegraded":"2024-04-23T01:57:51.594969+0000","last_fullsized":"2024-04-23T01:57:51.594969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":197,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.442198+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:57:51.442198+0000","last_peered":"2024-04-23T01:57:51.442198+0000","last_clean":"2024-04-23T01:57:51.442198+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:57:51.442198+0000","last_undegraded":"2024-04-23T01:57:51.442198+0000","last_fullsized":"2024-04-23T01:57:51.442198+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.660500+0000","last_change":"2024-04-23T01:57:51.662237+0000","last_active":"2024-04-23T01:57:51.660500+0000","last_peered":"2024-04-23T01:57:51.660500+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.647583+0000","last_became_peered":"2024-04-23T01:57:51.647583+0000","last_unstale":"2024-04-23T01:57:51.660500+0000","last_undegraded":"2024-04-23T01:57:51.660500+0000","last_fullsized":"2024-04-23T01:57:51.660500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":189,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.480887+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:57:51.480887+0000","last_peered":"2024-04-23T01:57:51.480887+0000","last_clean":"2024-04-23T01:57:51.480887+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:57:51.480887+0000","last_undegraded":"2024-04-23T01:57:51.480887+0000","last_fullsized":"2024-04-23T01:57:51.480887+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":7,"reported_epoch":83,"state":"active","last_fresh":"2024-04-23T01:57:51.670788+0000","last_change":"2024-04-23T01:57:51.670788+0000","last_active":"2024-04-23T01:57:51.670788+0000","last_peered":"2024-04-23T01:57:51.670788+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.670788+0000","last_became_peered":"2024-04-23T01:57:51.670788+0000","last_unstale":"2024-04-23T01:57:51.670788+0000","last_undegraded":"2024-04-23T01:57:51.670788+0000","last_fullsized":"2024-04-23T01:57:51.670788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":["0","2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":195,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.427075+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:57:51.427075+0000","last_peered":"2024-04-23T01:57:51.427075+0000","last_clean":"2024-04-23T01:57:51.427075+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:57:51.427075+0000","last_undegraded":"2024-04-23T01:57:51.427075+0000","last_fullsized":"2024-04-23T01:57:51.427075+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.630684+0000","last_change":"2024-04-23T01:57:51.632423+0000","last_active":"2024-04-23T01:57:51.630684+0000","last_peered":"2024-04-23T01:57:51.630684+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.624354+0000","last_became_peered":"2024-04-23T01:57:51.624354+0000","last_unstale":"2024-04-23T01:57:51.630684+0000","last_undegraded":"2024-04-23T01:57:51.630684+0000","last_fullsized":"2024-04-23T01:57:51.630684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":195,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.459889+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:57:51.459889+0000","last_peered":"2024-04-23T01:57:51.459889+0000","last_clean":"2024-04-23T01:57:51.459889+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:57:51.459889+0000","last_undegraded":"2024-04-23T01:57:51.459889+0000","last_fullsized":"2024-04-23T01:57:51.459889+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":4,"reported_epoch":83,"state":"creating+activating","last_fresh":"2024-04-23T01:57:51.436485+0000","last_change":"2024-04-23T01:57:51.436485+0000","last_active":"2024-04-23T01:57:50.388836+0000","last_peered":"2024-04-23T01:57:50.388836+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:57:51.436485+0000","last_undegraded":"2024-04-23T01:57:51.436485+0000","last_fullsized":"2024-04-23T01:57:51.436485+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":["0","1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":195,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.431160+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:57:51.431160+0000","last_peered":"2024-04-23T01:57:51.431160+0000","last_clean":"2024-04-23T01:57:51.431160+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:57:51.431160+0000","last_undegraded":"2024-04-23T01:57:51.431160+0000","last_fullsized":"2024-04-23T01:57:51.431160+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.655093+0000","last_change":"2024-04-23T01:57:51.656207+0000","last_active":"2024-04-23T01:57:51.655093+0000","last_peered":"2024-04-23T01:57:51.655093+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.638689+0000","last_became_peered":"2024-04-23T01:57:51.638689+0000","last_unstale":"2024-04-23T01:57:51.655093+0000","last_undegraded":"2024-04-23T01:57:51.655093+0000","last_fullsized":"2024-04-23T01:57:51.655093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":197,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.440088+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:57:51.440088+0000","last_peered":"2024-04-23T01:57:51.440088+0000","last_clean":"2024-04-23T01:57:51.440088+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:57:51.440088+0000","last_undegraded":"2024-04-23T01:57:51.440088+0000","last_fullsized":"2024-04-23T01:57:51.440088+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706428,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":126140,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":125376,"kb_avail":94245700,"statfs":{"total":96636764160,"available":96507596800,"internally_reserved":0,"allocated":774144,"data_stored":852532,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":128385024},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706428,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":108092,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":107328,"kb_avail":94263748,"statfs":{"total":96636764160,"available":96526077952,"internally_reserved":0,"allocated":774144,"data_stored":852532,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":109903872},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706428,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":108024,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":107328,"kb_avail":94263816,"statfs":{"total":96636764160,"available":96526147584,"internally_reserved":0,"allocated":704512,"data_stored":258547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":109903872},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706427,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":121972,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":121216,"kb_avail":94249868,"statfs":{"total":96636764160,"available":96511864832,"internally_reserved":0,"allocated":765952,"data_stored":848915,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":124125184},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:57:54.179 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:57:54.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:54.303+0000 7fbf7995d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:54.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:54.310+0000 7fbf7995d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:54.481 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 2 16 277 261 1.01941 1.23438 0.143209 0.117323 2024-04-23T01:57:55.481 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 3 16 438 422 1.09884 1.25781 0.0883165 0.111614 2024-04-23T01:57:56.483 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 4 16 598 582 1.1366 1.25 0.177376 0.108655 2024-04-23T01:57:56.495 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:57:56.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:56.615+0000 7fd076c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:56.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:56.622+0000 7fd076c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:57.146 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:57:57.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:57.256+0000 7fe6ffead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:57.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:57.265+0000 7fe6ffead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:57.481 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 5 16 738 722 1.12802 1.09375 0.224866 0.108828 2024-04-23T01:57:57.484 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:57:57.485 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:57:57.506 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":462,"stamp":"2024-04-23T01:57:55.698546+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":464228,"kb_used_data":2948,"kb_used_omap":0,"kb_used_meta":461248,"kb_avail":377023132,"statfs":{"total":386547056640,"available":386071687168,"internally_reserved":0,"allocated":3018752,"data_stored":2812526,"data_compressed":89859,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":472317952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000950"},"pg_stats":[{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.582523+0000","last_change":"2024-04-23T01:57:51.583793+0000","last_active":"2024-04-23T01:57:51.582523+0000","last_peered":"2024-04-23T01:57:51.582523+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.574869+0000","last_became_peered":"2024-04-23T01:57:51.574869+0000","last_unstale":"2024-04-23T01:57:51.582523+0000","last_undegraded":"2024-04-23T01:57:51.582523+0000","last_fullsized":"2024-04-23T01:57:51.582523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.678020+0000","last_change":"2024-04-23T01:57:51.679265+0000","last_active":"2024-04-23T01:57:51.678020+0000","last_peered":"2024-04-23T01:57:51.678020+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.662859+0000","last_became_peered":"2024-04-23T01:57:51.662859+0000","last_unstale":"2024-04-23T01:57:51.678020+0000","last_undegraded":"2024-04-23T01:57:51.678020+0000","last_fullsized":"2024-04-23T01:57:51.678020+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.592258+0000","last_change":"2024-04-23T01:57:51.593118+0000","last_active":"2024-04-23T01:57:51.592258+0000","last_peered":"2024-04-23T01:57:51.592258+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.580553+0000","last_became_peered":"2024-04-23T01:57:51.580553+0000","last_unstale":"2024-04-23T01:57:51.592258+0000","last_undegraded":"2024-04-23T01:57:51.592258+0000","last_fullsized":"2024-04-23T01:57:51.592258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.575835+0000","last_change":"2024-04-23T01:57:51.577069+0000","last_active":"2024-04-23T01:57:51.575835+0000","last_peered":"2024-04-23T01:57:51.575835+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.568227+0000","last_became_peered":"2024-04-23T01:57:51.568227+0000","last_unstale":"2024-04-23T01:57:51.575835+0000","last_undegraded":"2024-04-23T01:57:51.575835+0000","last_fullsized":"2024-04-23T01:57:51.575835+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":4,"reported_epoch":83,"state":"creating+activating","last_fresh":"2024-04-23T01:57:51.460986+0000","last_change":"2024-04-23T01:57:51.460986+0000","last_active":"2024-04-23T01:57:50.388836+0000","last_peered":"2024-04-23T01:57:50.388836+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:57:51.460986+0000","last_undegraded":"2024-04-23T01:57:51.460986+0000","last_fullsized":"2024-04-23T01:57:51.460986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":["0","2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.628393+0000","last_change":"2024-04-23T01:57:51.630095+0000","last_active":"2024-04-23T01:57:51.628393+0000","last_peered":"2024-04-23T01:57:51.628393+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.623783+0000","last_became_peered":"2024-04-23T01:57:51.623783+0000","last_unstale":"2024-04-23T01:57:51.628393+0000","last_undegraded":"2024-04-23T01:57:51.628393+0000","last_fullsized":"2024-04-23T01:57:51.628393+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.603232+0000","last_change":"2024-04-23T01:57:51.605374+0000","last_active":"2024-04-23T01:57:51.603232+0000","last_peered":"2024-04-23T01:57:51.603232+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.597053+0000","last_became_peered":"2024-04-23T01:57:51.597053+0000","last_unstale":"2024-04-23T01:57:51.603232+0000","last_undegraded":"2024-04-23T01:57:51.603232+0000","last_fullsized":"2024-04-23T01:57:51.603232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.640703+0000","last_change":"2024-04-23T01:57:51.641687+0000","last_active":"2024-04-23T01:57:51.640703+0000","last_peered":"2024-04-23T01:57:51.640703+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.623973+0000","last_became_peered":"2024-04-23T01:57:51.623973+0000","last_unstale":"2024-04-23T01:57:51.640703+0000","last_undegraded":"2024-04-23T01:57:51.640703+0000","last_fullsized":"2024-04-23T01:57:51.640703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.668799+0000","last_change":"2024-04-23T01:57:51.669918+0000","last_active":"2024-04-23T01:57:51.668799+0000","last_peered":"2024-04-23T01:57:51.668799+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.652610+0000","last_became_peered":"2024-04-23T01:57:51.652610+0000","last_unstale":"2024-04-23T01:57:51.668799+0000","last_undegraded":"2024-04-23T01:57:51.668799+0000","last_fullsized":"2024-04-23T01:57:51.668799+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":328,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.410996+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:57:51.410996+0000","last_peered":"2024-04-23T01:57:51.410996+0000","last_clean":"2024-04-23T01:57:51.410996+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:57:51.410996+0000","last_undegraded":"2024-04-23T01:57:51.410996+0000","last_fullsized":"2024-04-23T01:57:51.410996+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":197,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.435401+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:57:51.435401+0000","last_peered":"2024-04-23T01:57:51.435401+0000","last_clean":"2024-04-23T01:57:51.435401+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:57:51.435401+0000","last_undegraded":"2024-04-23T01:57:51.435401+0000","last_fullsized":"2024-04-23T01:57:51.435401+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.586325+0000","last_change":"2024-04-23T01:57:51.587471+0000","last_active":"2024-04-23T01:57:51.586325+0000","last_peered":"2024-04-23T01:57:51.586325+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.575352+0000","last_became_peered":"2024-04-23T01:57:51.575352+0000","last_unstale":"2024-04-23T01:57:51.586325+0000","last_undegraded":"2024-04-23T01:57:51.586325+0000","last_fullsized":"2024-04-23T01:57:51.586325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":189,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.475424+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:57:51.475424+0000","last_peered":"2024-04-23T01:57:51.475424+0000","last_clean":"2024-04-23T01:57:51.475424+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:57:51.475424+0000","last_undegraded":"2024-04-23T01:57:51.475424+0000","last_fullsized":"2024-04-23T01:57:51.475424+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.594969+0000","last_change":"2024-04-23T01:57:51.596134+0000","last_active":"2024-04-23T01:57:51.594969+0000","last_peered":"2024-04-23T01:57:51.594969+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.589605+0000","last_became_peered":"2024-04-23T01:57:51.589605+0000","last_unstale":"2024-04-23T01:57:51.594969+0000","last_undegraded":"2024-04-23T01:57:51.594969+0000","last_fullsized":"2024-04-23T01:57:51.594969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":197,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.442198+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:57:51.442198+0000","last_peered":"2024-04-23T01:57:51.442198+0000","last_clean":"2024-04-23T01:57:51.442198+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:57:51.442198+0000","last_undegraded":"2024-04-23T01:57:51.442198+0000","last_fullsized":"2024-04-23T01:57:51.442198+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.660500+0000","last_change":"2024-04-23T01:57:51.662237+0000","last_active":"2024-04-23T01:57:51.660500+0000","last_peered":"2024-04-23T01:57:51.660500+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.647583+0000","last_became_peered":"2024-04-23T01:57:51.647583+0000","last_unstale":"2024-04-23T01:57:51.660500+0000","last_undegraded":"2024-04-23T01:57:51.660500+0000","last_fullsized":"2024-04-23T01:57:51.660500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":189,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.480887+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:57:51.480887+0000","last_peered":"2024-04-23T01:57:51.480887+0000","last_clean":"2024-04-23T01:57:51.480887+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:57:51.480887+0000","last_undegraded":"2024-04-23T01:57:51.480887+0000","last_fullsized":"2024-04-23T01:57:51.480887+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":7,"reported_epoch":83,"state":"active","last_fresh":"2024-04-23T01:57:51.670788+0000","last_change":"2024-04-23T01:57:51.670788+0000","last_active":"2024-04-23T01:57:51.670788+0000","last_peered":"2024-04-23T01:57:51.670788+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.670788+0000","last_became_peered":"2024-04-23T01:57:51.670788+0000","last_unstale":"2024-04-23T01:57:51.670788+0000","last_undegraded":"2024-04-23T01:57:51.670788+0000","last_fullsized":"2024-04-23T01:57:51.670788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":["0","2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":195,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.427075+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:57:51.427075+0000","last_peered":"2024-04-23T01:57:51.427075+0000","last_clean":"2024-04-23T01:57:51.427075+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:57:51.427075+0000","last_undegraded":"2024-04-23T01:57:51.427075+0000","last_fullsized":"2024-04-23T01:57:51.427075+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.630684+0000","last_change":"2024-04-23T01:57:51.632423+0000","last_active":"2024-04-23T01:57:51.630684+0000","last_peered":"2024-04-23T01:57:51.630684+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.624354+0000","last_became_peered":"2024-04-23T01:57:51.624354+0000","last_unstale":"2024-04-23T01:57:51.630684+0000","last_undegraded":"2024-04-23T01:57:51.630684+0000","last_fullsized":"2024-04-23T01:57:51.630684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":195,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.459889+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:57:51.459889+0000","last_peered":"2024-04-23T01:57:51.459889+0000","last_clean":"2024-04-23T01:57:51.459889+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:57:51.459889+0000","last_undegraded":"2024-04-23T01:57:51.459889+0000","last_fullsized":"2024-04-23T01:57:51.459889+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":4,"reported_epoch":83,"state":"creating+activating","last_fresh":"2024-04-23T01:57:51.436485+0000","last_change":"2024-04-23T01:57:51.436485+0000","last_active":"2024-04-23T01:57:50.388836+0000","last_peered":"2024-04-23T01:57:50.388836+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T01:57:51.436485+0000","last_undegraded":"2024-04-23T01:57:51.436485+0000","last_fullsized":"2024-04-23T01:57:51.436485+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":["0","1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":195,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.431160+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:57:51.431160+0000","last_peered":"2024-04-23T01:57:51.431160+0000","last_clean":"2024-04-23T01:57:51.431160+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:57:51.431160+0000","last_undegraded":"2024-04-23T01:57:51.431160+0000","last_fullsized":"2024-04-23T01:57:51.431160+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.655093+0000","last_change":"2024-04-23T01:57:51.656207+0000","last_active":"2024-04-23T01:57:51.655093+0000","last_peered":"2024-04-23T01:57:51.655093+0000","last_clean":"2024-04-23T01:57:50.388836+0000","last_became_active":"2024-04-23T01:57:51.638689+0000","last_became_peered":"2024-04-23T01:57:51.638689+0000","last_unstale":"2024-04-23T01:57:51.655093+0000","last_undegraded":"2024-04-23T01:57:51.655093+0000","last_fullsized":"2024-04-23T01:57:51.655093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":197,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T01:57:51.440088+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:57:51.440088+0000","last_peered":"2024-04-23T01:57:51.440088+0000","last_clean":"2024-04-23T01:57:51.440088+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:57:51.440088+0000","last_undegraded":"2024-04-23T01:57:51.440088+0000","last_fullsized":"2024-04-23T01:57:51.440088+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706428,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":126140,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":125376,"kb_avail":94245700,"statfs":{"total":96636764160,"available":96507596800,"internally_reserved":0,"allocated":774144,"data_stored":852532,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":128385024},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706428,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":108092,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":107328,"kb_avail":94263748,"statfs":{"total":96636764160,"available":96526077952,"internally_reserved":0,"allocated":774144,"data_stored":852532,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":109903872},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706428,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":108024,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":107328,"kb_avail":94263816,"statfs":{"total":96636764160,"available":96526147584,"internally_reserved":0,"allocated":704512,"data_stored":258547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":109903872},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706427,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":121972,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":121216,"kb_avail":94249868,"statfs":{"total":96636764160,"available":96511864832,"internally_reserved":0,"allocated":765952,"data_stored":848915,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":124125184},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:57:57.507 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-23T01:57:57.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:57.623+0000 7f270139d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:57.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:57.630+0000 7f270139d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:57.938 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:57:57.939 INFO:teuthology.orchestra.run.smithi090.stdout:{"fsid":"0a731bfd-0fc4-4c1b-8e4b-f6dfbc059bcc","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":778,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":84,"num_osds":4,"num_up_osds":4,"osd_up_since":1713836706,"num_in_osds":4,"osd_in_since":1713836701,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"creating+activating","count":2},{"state_name":"active","count":1}],"num_pgs":25,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":475369472,"bytes_avail":386071687168,"bytes_total":386547056640,"inactive_pgs_ratio":0.079999998211860657},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-04-23T01:46:05.559282+0000","services":{}},"progress_events":{}} 2024-04-23T01:57:57.955 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-04-23T01:57:57.955 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:57:58.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:58.074+0000 7f2efdd5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:58.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:58.085+0000 7f2efdd5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:58.464 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:57:58.465 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:57:58.479 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2024-04-23T01:57:57.698908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5653024,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":755,"num_write_kb":6838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":699,"ondisk_log_size":699,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":479028,"kb_used_data":9428,"kb_used_omap":0,"kb_used_meta":469568,"kb_avail":377008332,"statfs":{"total":386547056640,"available":386056531968,"internally_reserved":0,"allocated":9654272,"data_stored":16006836,"data_compressed":768105,"data_compressed_allocated":6709248,"data_compressed_original":14893056,"omap_allocated":0,"internal_metadata":480837632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5062656,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":618,"num_write_kb":4944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.303608"},"pg_stats":[{"pgid":"10.7","version":"84'23","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:55.908062+0000","last_change":"2024-04-23T01:57:51.583793+0000","last_active":"2024-04-23T01:57:55.908062+0000","last_peered":"2024-04-23T01:57:55.908062+0000","last_clean":"2024-04-23T01:57:55.908062+0000","last_became_active":"2024-04-23T01:57:51.574869+0000","last_became_peered":"2024-04-23T01:57:51.574869+0000","last_unstale":"2024-04-23T01:57:55.908062+0000","last_undegraded":"2024-04-23T01:57:55.908062+0000","last_fullsized":"2024-04-23T01:57:55.908062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"84'31","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:54.561761+0000","last_change":"2024-04-23T01:57:51.679265+0000","last_active":"2024-04-23T01:57:54.561761+0000","last_peered":"2024-04-23T01:57:54.561761+0000","last_clean":"2024-04-23T01:57:54.561761+0000","last_became_active":"2024-04-23T01:57:51.662859+0000","last_became_peered":"2024-04-23T01:57:51.662859+0000","last_unstale":"2024-04-23T01:57:54.561761+0000","last_undegraded":"2024-04-23T01:57:54.561761+0000","last_fullsized":"2024-04-23T01:57:54.561761+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"84'57","reported_seq":70,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:56.727969+0000","last_change":"2024-04-23T01:57:51.593118+0000","last_active":"2024-04-23T01:57:56.727969+0000","last_peered":"2024-04-23T01:57:56.727969+0000","last_clean":"2024-04-23T01:57:56.727969+0000","last_became_active":"2024-04-23T01:57:51.580553+0000","last_became_peered":"2024-04-23T01:57:51.580553+0000","last_unstale":"2024-04-23T01:57:56.727969+0000","last_undegraded":"2024-04-23T01:57:56.727969+0000","last_fullsized":"2024-04-23T01:57:56.727969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":475136,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58,"num_write_kb":464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"84'15","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:56.231476+0000","last_change":"2024-04-23T01:57:51.577069+0000","last_active":"2024-04-23T01:57:56.231476+0000","last_peered":"2024-04-23T01:57:56.231476+0000","last_clean":"2024-04-23T01:57:56.231476+0000","last_became_active":"2024-04-23T01:57:51.568227+0000","last_became_peered":"2024-04-23T01:57:51.568227+0000","last_unstale":"2024-04-23T01:57:56.231476+0000","last_undegraded":"2024-04-23T01:57:56.231476+0000","last_fullsized":"2024-04-23T01:57:56.231476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"84'47","reported_seq":60,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:56.646028+0000","last_change":"2024-04-23T01:57:51.748592+0000","last_active":"2024-04-23T01:57:56.646028+0000","last_peered":"2024-04-23T01:57:56.646028+0000","last_clean":"2024-04-23T01:57:56.646028+0000","last_became_active":"2024-04-23T01:57:51.742159+0000","last_became_peered":"2024-04-23T01:57:51.742159+0000","last_unstale":"2024-04-23T01:57:56.646028+0000","last_undegraded":"2024-04-23T01:57:56.646028+0000","last_fullsized":"2024-04-23T01:57:56.646028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"84'39","reported_seq":52,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:56.525809+0000","last_change":"2024-04-23T01:57:51.630095+0000","last_active":"2024-04-23T01:57:56.525809+0000","last_peered":"2024-04-23T01:57:56.525809+0000","last_clean":"2024-04-23T01:57:56.525809+0000","last_became_active":"2024-04-23T01:57:51.623783+0000","last_became_peered":"2024-04-23T01:57:51.623783+0000","last_unstale":"2024-04-23T01:57:56.525809+0000","last_undegraded":"2024-04-23T01:57:56.525809+0000","last_fullsized":"2024-04-23T01:57:56.525809+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"84'31","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:56.217709+0000","last_change":"2024-04-23T01:57:51.605374+0000","last_active":"2024-04-23T01:57:56.217709+0000","last_peered":"2024-04-23T01:57:56.217709+0000","last_clean":"2024-04-23T01:57:56.217709+0000","last_became_active":"2024-04-23T01:57:51.597053+0000","last_became_peered":"2024-04-23T01:57:51.597053+0000","last_unstale":"2024-04-23T01:57:56.217709+0000","last_undegraded":"2024-04-23T01:57:56.217709+0000","last_fullsized":"2024-04-23T01:57:56.217709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"84'47","reported_seq":60,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:56.063329+0000","last_change":"2024-04-23T01:57:51.641687+0000","last_active":"2024-04-23T01:57:56.063329+0000","last_peered":"2024-04-23T01:57:56.063329+0000","last_clean":"2024-04-23T01:57:56.063329+0000","last_became_active":"2024-04-23T01:57:51.623973+0000","last_became_peered":"2024-04-23T01:57:51.623973+0000","last_unstale":"2024-04-23T01:57:56.063329+0000","last_undegraded":"2024-04-23T01:57:56.063329+0000","last_fullsized":"2024-04-23T01:57:56.063329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"84'47","reported_seq":60,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:56.597244+0000","last_change":"2024-04-23T01:57:51.669918+0000","last_active":"2024-04-23T01:57:56.597244+0000","last_peered":"2024-04-23T01:57:56.597244+0000","last_clean":"2024-04-23T01:57:56.597244+0000","last_became_active":"2024-04-23T01:57:51.652610+0000","last_became_peered":"2024-04-23T01:57:51.652610+0000","last_unstale":"2024-04-23T01:57:56.597244+0000","last_undegraded":"2024-04-23T01:57:56.597244+0000","last_fullsized":"2024-04-23T01:57:56.597244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":331,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:52.416961+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:57:52.416961+0000","last_peered":"2024-04-23T01:57:52.416961+0000","last_clean":"2024-04-23T01:57:52.416961+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:57:52.416961+0000","last_undegraded":"2024-04-23T01:57:52.416961+0000","last_fullsized":"2024-04-23T01:57:52.416961+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":200,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:52.527962+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:57:52.527962+0000","last_peered":"2024-04-23T01:57:52.527962+0000","last_clean":"2024-04-23T01:57:52.527962+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:57:52.527962+0000","last_undegraded":"2024-04-23T01:57:52.527962+0000","last_fullsized":"2024-04-23T01:57:52.527962+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"84'31","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:55.844213+0000","last_change":"2024-04-23T01:57:51.587471+0000","last_active":"2024-04-23T01:57:55.844213+0000","last_peered":"2024-04-23T01:57:55.844213+0000","last_clean":"2024-04-23T01:57:55.844213+0000","last_became_active":"2024-04-23T01:57:51.575352+0000","last_became_peered":"2024-04-23T01:57:51.575352+0000","last_unstale":"2024-04-23T01:57:55.844213+0000","last_undegraded":"2024-04-23T01:57:55.844213+0000","last_fullsized":"2024-04-23T01:57:55.844213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":192,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:52.589906+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:57:52.589906+0000","last_peered":"2024-04-23T01:57:52.589906+0000","last_clean":"2024-04-23T01:57:52.589906+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:57:52.589906+0000","last_undegraded":"2024-04-23T01:57:52.589906+0000","last_fullsized":"2024-04-23T01:57:52.589906+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"84'47","reported_seq":60,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:55.840253+0000","last_change":"2024-04-23T01:57:51.596134+0000","last_active":"2024-04-23T01:57:55.840253+0000","last_peered":"2024-04-23T01:57:55.840253+0000","last_clean":"2024-04-23T01:57:55.840253+0000","last_became_active":"2024-04-23T01:57:51.589605+0000","last_became_peered":"2024-04-23T01:57:51.589605+0000","last_unstale":"2024-04-23T01:57:55.840253+0000","last_undegraded":"2024-04-23T01:57:55.840253+0000","last_fullsized":"2024-04-23T01:57:55.840253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":200,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:52.519415+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:57:52.519415+0000","last_peered":"2024-04-23T01:57:52.519415+0000","last_clean":"2024-04-23T01:57:52.519415+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:57:52.519415+0000","last_undegraded":"2024-04-23T01:57:52.519415+0000","last_fullsized":"2024-04-23T01:57:52.519415+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"84'23","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:55.757428+0000","last_change":"2024-04-23T01:57:51.662237+0000","last_active":"2024-04-23T01:57:55.757428+0000","last_peered":"2024-04-23T01:57:55.757428+0000","last_clean":"2024-04-23T01:57:55.757428+0000","last_became_active":"2024-04-23T01:57:51.647583+0000","last_became_peered":"2024-04-23T01:57:51.647583+0000","last_unstale":"2024-04-23T01:57:55.757428+0000","last_undegraded":"2024-04-23T01:57:55.757428+0000","last_fullsized":"2024-04-23T01:57:55.757428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":192,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:52.620913+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:57:52.620913+0000","last_peered":"2024-04-23T01:57:52.620913+0000","last_clean":"2024-04-23T01:57:52.620913+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:57:52.620913+0000","last_undegraded":"2024-04-23T01:57:52.620913+0000","last_fullsized":"2024-04-23T01:57:52.620913+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"84'39","reported_seq":52,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:56.529849+0000","last_change":"2024-04-23T01:57:51.681957+0000","last_active":"2024-04-23T01:57:56.529849+0000","last_peered":"2024-04-23T01:57:56.529849+0000","last_clean":"2024-04-23T01:57:56.529849+0000","last_became_active":"2024-04-23T01:57:51.670788+0000","last_became_peered":"2024-04-23T01:57:51.670788+0000","last_unstale":"2024-04-23T01:57:56.529849+0000","last_undegraded":"2024-04-23T01:57:56.529849+0000","last_fullsized":"2024-04-23T01:57:56.529849+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":198,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:53.340562+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:57:53.340562+0000","last_peered":"2024-04-23T01:57:53.340562+0000","last_clean":"2024-04-23T01:57:53.340562+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:57:53.340562+0000","last_undegraded":"2024-04-23T01:57:53.340562+0000","last_fullsized":"2024-04-23T01:57:53.340562+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"84'67","reported_seq":80,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:56.639602+0000","last_change":"2024-04-23T01:57:51.632423+0000","last_active":"2024-04-23T01:57:56.639602+0000","last_peered":"2024-04-23T01:57:56.639602+0000","last_clean":"2024-04-23T01:57:56.639602+0000","last_became_active":"2024-04-23T01:57:51.624354+0000","last_became_peered":"2024-04-23T01:57:51.624354+0000","last_unstale":"2024-04-23T01:57:56.639602+0000","last_undegraded":"2024-04-23T01:57:56.639602+0000","last_fullsized":"2024-04-23T01:57:56.639602+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":557056,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":198,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:52.442219+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:57:52.442219+0000","last_peered":"2024-04-23T01:57:52.442219+0000","last_clean":"2024-04-23T01:57:52.442219+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:57:52.442219+0000","last_undegraded":"2024-04-23T01:57:52.442219+0000","last_fullsized":"2024-04-23T01:57:52.442219+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"84'39","reported_seq":52,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:55.115825+0000","last_change":"2024-04-23T01:57:51.735746+0000","last_active":"2024-04-23T01:57:55.115825+0000","last_peered":"2024-04-23T01:57:55.115825+0000","last_clean":"2024-04-23T01:57:55.115825+0000","last_became_active":"2024-04-23T01:57:51.728785+0000","last_became_peered":"2024-04-23T01:57:51.728785+0000","last_unstale":"2024-04-23T01:57:55.115825+0000","last_undegraded":"2024-04-23T01:57:55.115825+0000","last_fullsized":"2024-04-23T01:57:55.115825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":198,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:53.355136+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:57:53.355136+0000","last_peered":"2024-04-23T01:57:53.355136+0000","last_clean":"2024-04-23T01:57:53.355136+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:57:53.355136+0000","last_undegraded":"2024-04-23T01:57:53.355136+0000","last_fullsized":"2024-04-23T01:57:53.355136+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"84'19","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:56.675952+0000","last_change":"2024-04-23T01:57:51.656207+0000","last_active":"2024-04-23T01:57:56.675952+0000","last_peered":"2024-04-23T01:57:56.675952+0000","last_clean":"2024-04-23T01:57:56.675952+0000","last_became_active":"2024-04-23T01:57:51.638689+0000","last_became_peered":"2024-04-23T01:57:51.638689+0000","last_unstale":"2024-04-23T01:57:56.675952+0000","last_undegraded":"2024-04-23T01:57:56.675952+0000","last_fullsized":"2024-04-23T01:57:56.675952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163840,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":200,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:52.522373+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:57:52.522373+0000","last_peered":"2024-04-23T01:57:52.522373+0000","last_clean":"2024-04-23T01:57:52.522373+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:57:52.522373+0000","last_undegraded":"2024-04-23T01:57:52.522373+0000","last_fullsized":"2024-04-23T01:57:52.522373+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":5062656,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":618,"num_write_kb":4944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":7614464,"data_stored":15228928,"data_compressed":784168,"data_compressed_allocated":7614464,"data_compressed_original":15228928,"omap_allocated":0,"internal_metadata":0},"log_size":602,"ondisk_log_size":602,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706433,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":131716,"kb_used_data":2172,"kb_used_omap":0,"kb_used_meta":129536,"kb_avail":94240124,"statfs":{"total":96636764160,"available":96501886976,"internally_reserved":0,"allocated":2224128,"data_stored":3739701,"data_compressed":178437,"data_compressed_allocated":1482752,"data_compressed_original":3457024,"omap_allocated":0,"internal_metadata":132644864},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706433,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":110104,"kb_used_data":2768,"kb_used_omap":0,"kb_used_meta":107328,"kb_avail":94261736,"statfs":{"total":96636764160,"available":96524017664,"internally_reserved":0,"allocated":2834432,"data_stored":4960309,"data_compressed":241273,"data_compressed_allocated":2093056,"data_compressed_original":4677632,"omap_allocated":0,"internal_metadata":109903872},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706432,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":113704,"kb_used_data":2208,"kb_used_omap":0,"kb_used_meta":111488,"kb_avail":94258136,"statfs":{"total":96636764160,"available":96520331264,"internally_reserved":0,"allocated":2260992,"data_stored":3345941,"data_compressed":158578,"data_compressed_allocated":1540096,"data_compressed_original":3080192,"omap_allocated":0,"internal_metadata":114163712},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706432,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":123504,"kb_used_data":2280,"kb_used_omap":0,"kb_used_meta":121216,"kb_avail":94248336,"statfs":{"total":96636764160,"available":96510296064,"internally_reserved":0,"allocated":2334720,"data_stored":3960885,"data_compressed":189817,"data_compressed_allocated":1593344,"data_compressed_original":3678208,"omap_allocated":0,"internal_metadata":124125184},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":4055040,"data_compressed":208816,"data_compressed_allocated":2027520,"data_compressed_original":4055040,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1937408,"data_stored":3874816,"data_compressed":199512,"data_compressed_allocated":1937408,"data_compressed_original":3874816,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2203648,"data_stored":4407296,"data_compressed":226934,"data_compressed_allocated":2203648,"data_compressed_original":4407296,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1445888,"data_stored":2891776,"data_compressed":148906,"data_compressed_allocated":1445888,"data_compressed_original":2891776,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:57:58.481 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 6 16 881 865 1.12619 1.11719 0.244661 0.10995 2024-04-23T01:57:58.804 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:57:58.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:58.931+0000 7fe9d2f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:58.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:57:58.939+0000 7fe9d2f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:57:59.481 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 7 16 1028 1012 1.12936 1.14844 0.223819 0.109241 2024-04-23T01:58:00.481 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 8 16 1165 1149 1.12197 1.07031 0.0676165 0.110589 2024-04-23T01:58:01.091 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:58:01.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:01.219+0000 7fb1136ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:01.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:01.226+0000 7fb1136ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:01.481 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:58:01.482 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 9 16 1324 1308 1.13531 1.24219 0.126691 0.109379 2024-04-23T01:58:01.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:01.621+0000 7fb0967ff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:01.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:01.629+0000 7fb0967ff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:01.852 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:58:01.852 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:58:01.870 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2024-04-23T01:58:01.699823+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8241696,"num_objects":120,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":1071,"num_write_kb":9366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1015,"ondisk_log_size":1015,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":485568,"kb_used_data":11808,"kb_used_omap":0,"kb_used_meta":473728,"kb_avail":377001792,"statfs":{"total":386547056640,"available":386049835008,"internally_reserved":0,"allocated":12091392,"data_stored":20881076,"data_compressed":1019385,"data_compressed_allocated":9146368,"data_compressed_original":19767296,"omap_allocated":0,"internal_metadata":485097472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7651328,"num_objects":118,"num_object_clones":0,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":934,"num_write_kb":7472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001951"},"pg_stats":[{"pgid":"10.7","version":"84'71","reported_seq":84,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:58:00.559083+0000","last_change":"2024-04-23T01:57:51.583793+0000","last_active":"2024-04-23T01:58:00.559083+0000","last_peered":"2024-04-23T01:58:00.559083+0000","last_clean":"2024-04-23T01:58:00.559083+0000","last_became_active":"2024-04-23T01:57:51.574869+0000","last_became_peered":"2024-04-23T01:57:51.574869+0000","last_unstale":"2024-04-23T01:58:00.559083+0000","last_undegraded":"2024-04-23T01:58:00.559083+0000","last_fullsized":"2024-04-23T01:58:00.559083+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"84'31","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:54.561761+0000","last_change":"2024-04-23T01:57:51.679265+0000","last_active":"2024-04-23T01:57:54.561761+0000","last_peered":"2024-04-23T01:57:54.561761+0000","last_clean":"2024-04-23T01:57:54.561761+0000","last_became_active":"2024-04-23T01:57:51.662859+0000","last_became_peered":"2024-04-23T01:57:51.662859+0000","last_unstale":"2024-04-23T01:57:54.561761+0000","last_undegraded":"2024-04-23T01:57:54.561761+0000","last_fullsized":"2024-04-23T01:57:54.561761+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"84'57","reported_seq":70,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:56.727969+0000","last_change":"2024-04-23T01:57:51.593118+0000","last_active":"2024-04-23T01:57:56.727969+0000","last_peered":"2024-04-23T01:57:56.727969+0000","last_clean":"2024-04-23T01:57:56.727969+0000","last_became_active":"2024-04-23T01:57:51.580553+0000","last_became_peered":"2024-04-23T01:57:51.580553+0000","last_unstale":"2024-04-23T01:57:56.727969+0000","last_undegraded":"2024-04-23T01:57:56.727969+0000","last_fullsized":"2024-04-23T01:57:56.727969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":475136,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58,"num_write_kb":464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"84'15","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:56.231476+0000","last_change":"2024-04-23T01:57:51.577069+0000","last_active":"2024-04-23T01:57:56.231476+0000","last_peered":"2024-04-23T01:57:56.231476+0000","last_clean":"2024-04-23T01:57:56.231476+0000","last_became_active":"2024-04-23T01:57:51.568227+0000","last_became_peered":"2024-04-23T01:57:51.568227+0000","last_unstale":"2024-04-23T01:57:56.231476+0000","last_undegraded":"2024-04-23T01:57:56.231476+0000","last_fullsized":"2024-04-23T01:57:56.231476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"84'87","reported_seq":100,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:58:01.456659+0000","last_change":"2024-04-23T01:57:51.748592+0000","last_active":"2024-04-23T01:58:01.456659+0000","last_peered":"2024-04-23T01:58:01.456659+0000","last_clean":"2024-04-23T01:58:01.456659+0000","last_became_active":"2024-04-23T01:57:51.742159+0000","last_became_peered":"2024-04-23T01:57:51.742159+0000","last_unstale":"2024-04-23T01:58:01.456659+0000","last_undegraded":"2024-04-23T01:58:01.456659+0000","last_fullsized":"2024-04-23T01:58:01.456659+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":720896,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"84'39","reported_seq":52,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:56.525809+0000","last_change":"2024-04-23T01:57:51.630095+0000","last_active":"2024-04-23T01:57:56.525809+0000","last_peered":"2024-04-23T01:57:56.525809+0000","last_clean":"2024-04-23T01:57:56.525809+0000","last_became_active":"2024-04-23T01:57:51.623783+0000","last_became_peered":"2024-04-23T01:57:51.623783+0000","last_unstale":"2024-04-23T01:57:56.525809+0000","last_undegraded":"2024-04-23T01:57:56.525809+0000","last_fullsized":"2024-04-23T01:57:56.525809+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"84'63","reported_seq":76,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:58:01.242686+0000","last_change":"2024-04-23T01:57:51.605374+0000","last_active":"2024-04-23T01:58:01.242686+0000","last_peered":"2024-04-23T01:58:01.242686+0000","last_clean":"2024-04-23T01:58:01.242686+0000","last_became_active":"2024-04-23T01:57:51.597053+0000","last_became_peered":"2024-04-23T01:57:51.597053+0000","last_unstale":"2024-04-23T01:58:01.242686+0000","last_undegraded":"2024-04-23T01:58:01.242686+0000","last_fullsized":"2024-04-23T01:58:01.242686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"84'47","reported_seq":60,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:56.063329+0000","last_change":"2024-04-23T01:57:51.641687+0000","last_active":"2024-04-23T01:57:56.063329+0000","last_peered":"2024-04-23T01:57:56.063329+0000","last_clean":"2024-04-23T01:57:56.063329+0000","last_became_active":"2024-04-23T01:57:51.623973+0000","last_became_peered":"2024-04-23T01:57:51.623973+0000","last_unstale":"2024-04-23T01:57:56.063329+0000","last_undegraded":"2024-04-23T01:57:56.063329+0000","last_fullsized":"2024-04-23T01:57:56.063329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"84'47","reported_seq":60,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:56.597244+0000","last_change":"2024-04-23T01:57:51.669918+0000","last_active":"2024-04-23T01:57:56.597244+0000","last_peered":"2024-04-23T01:57:56.597244+0000","last_clean":"2024-04-23T01:57:56.597244+0000","last_became_active":"2024-04-23T01:57:51.652610+0000","last_became_peered":"2024-04-23T01:57:51.652610+0000","last_unstale":"2024-04-23T01:57:56.597244+0000","last_undegraded":"2024-04-23T01:57:56.597244+0000","last_fullsized":"2024-04-23T01:57:56.597244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":331,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:52.416961+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:57:52.416961+0000","last_peered":"2024-04-23T01:57:52.416961+0000","last_clean":"2024-04-23T01:57:52.416961+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:57:52.416961+0000","last_undegraded":"2024-04-23T01:57:52.416961+0000","last_fullsized":"2024-04-23T01:57:52.416961+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":200,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:52.527962+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:57:52.527962+0000","last_peered":"2024-04-23T01:57:52.527962+0000","last_clean":"2024-04-23T01:57:52.527962+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:57:52.527962+0000","last_undegraded":"2024-04-23T01:57:52.527962+0000","last_fullsized":"2024-04-23T01:57:52.527962+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"84'31","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:55.844213+0000","last_change":"2024-04-23T01:57:51.587471+0000","last_active":"2024-04-23T01:57:55.844213+0000","last_peered":"2024-04-23T01:57:55.844213+0000","last_clean":"2024-04-23T01:57:55.844213+0000","last_became_active":"2024-04-23T01:57:51.575352+0000","last_became_peered":"2024-04-23T01:57:51.575352+0000","last_unstale":"2024-04-23T01:57:55.844213+0000","last_undegraded":"2024-04-23T01:57:55.844213+0000","last_fullsized":"2024-04-23T01:57:55.844213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":192,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:52.589906+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:57:52.589906+0000","last_peered":"2024-04-23T01:57:52.589906+0000","last_clean":"2024-04-23T01:57:52.589906+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:57:52.589906+0000","last_undegraded":"2024-04-23T01:57:52.589906+0000","last_fullsized":"2024-04-23T01:57:52.589906+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"84'47","reported_seq":60,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:55.840253+0000","last_change":"2024-04-23T01:57:51.596134+0000","last_active":"2024-04-23T01:57:55.840253+0000","last_peered":"2024-04-23T01:57:55.840253+0000","last_clean":"2024-04-23T01:57:55.840253+0000","last_became_active":"2024-04-23T01:57:51.589605+0000","last_became_peered":"2024-04-23T01:57:51.589605+0000","last_unstale":"2024-04-23T01:57:55.840253+0000","last_undegraded":"2024-04-23T01:57:55.840253+0000","last_fullsized":"2024-04-23T01:57:55.840253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":200,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:52.519415+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:57:52.519415+0000","last_peered":"2024-04-23T01:57:52.519415+0000","last_clean":"2024-04-23T01:57:52.519415+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:57:52.519415+0000","last_undegraded":"2024-04-23T01:57:52.519415+0000","last_fullsized":"2024-04-23T01:57:52.519415+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"84'71","reported_seq":84,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:58:01.557809+0000","last_change":"2024-04-23T01:57:51.662237+0000","last_active":"2024-04-23T01:58:01.557809+0000","last_peered":"2024-04-23T01:58:01.557809+0000","last_clean":"2024-04-23T01:58:01.557809+0000","last_became_active":"2024-04-23T01:57:51.647583+0000","last_became_peered":"2024-04-23T01:57:51.647583+0000","last_unstale":"2024-04-23T01:58:01.557809+0000","last_undegraded":"2024-04-23T01:58:01.557809+0000","last_fullsized":"2024-04-23T01:58:01.557809+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":192,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:52.620913+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:57:52.620913+0000","last_peered":"2024-04-23T01:57:52.620913+0000","last_clean":"2024-04-23T01:57:52.620913+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:57:52.620913+0000","last_undegraded":"2024-04-23T01:57:52.620913+0000","last_fullsized":"2024-04-23T01:57:52.620913+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"84'79","reported_seq":92,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:58:01.071312+0000","last_change":"2024-04-23T01:57:51.681957+0000","last_active":"2024-04-23T01:58:01.071312+0000","last_peered":"2024-04-23T01:58:01.071312+0000","last_clean":"2024-04-23T01:58:01.071312+0000","last_became_active":"2024-04-23T01:57:51.670788+0000","last_became_peered":"2024-04-23T01:57:51.670788+0000","last_unstale":"2024-04-23T01:58:01.071312+0000","last_undegraded":"2024-04-23T01:58:01.071312+0000","last_fullsized":"2024-04-23T01:58:01.071312+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":198,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:53.340562+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:57:53.340562+0000","last_peered":"2024-04-23T01:57:53.340562+0000","last_clean":"2024-04-23T01:57:53.340562+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:57:53.340562+0000","last_undegraded":"2024-04-23T01:57:53.340562+0000","last_fullsized":"2024-04-23T01:57:53.340562+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"84'159","reported_seq":172,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:58:01.395407+0000","last_change":"2024-04-23T01:57:51.632423+0000","last_active":"2024-04-23T01:58:01.395407+0000","last_peered":"2024-04-23T01:58:01.395407+0000","last_clean":"2024-04-23T01:58:01.395407+0000","last_became_active":"2024-04-23T01:57:51.624354+0000","last_became_peered":"2024-04-23T01:57:51.624354+0000","last_unstale":"2024-04-23T01:58:01.395407+0000","last_undegraded":"2024-04-23T01:58:01.395407+0000","last_fullsized":"2024-04-23T01:58:01.395407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310720,"num_objects":20,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":160,"num_write_kb":1280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":198,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:52.442219+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:57:52.442219+0000","last_peered":"2024-04-23T01:57:52.442219+0000","last_clean":"2024-04-23T01:57:52.442219+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:57:52.442219+0000","last_undegraded":"2024-04-23T01:57:52.442219+0000","last_fullsized":"2024-04-23T01:57:52.442219+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"84'55","reported_seq":68,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:58:00.229556+0000","last_change":"2024-04-23T01:57:51.735746+0000","last_active":"2024-04-23T01:58:00.229556+0000","last_peered":"2024-04-23T01:58:00.229556+0000","last_clean":"2024-04-23T01:58:00.229556+0000","last_became_active":"2024-04-23T01:57:51.728785+0000","last_became_peered":"2024-04-23T01:57:51.728785+0000","last_unstale":"2024-04-23T01:58:00.229556+0000","last_undegraded":"2024-04-23T01:58:00.229556+0000","last_fullsized":"2024-04-23T01:58:00.229556+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458752,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":198,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:53.355136+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:57:53.355136+0000","last_peered":"2024-04-23T01:57:53.355136+0000","last_clean":"2024-04-23T01:57:53.355136+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:57:53.355136+0000","last_undegraded":"2024-04-23T01:57:53.355136+0000","last_fullsized":"2024-04-23T01:57:53.355136+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"84'19","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:56.675952+0000","last_change":"2024-04-23T01:57:51.656207+0000","last_active":"2024-04-23T01:57:56.675952+0000","last_peered":"2024-04-23T01:57:56.675952+0000","last_clean":"2024-04-23T01:57:56.675952+0000","last_became_active":"2024-04-23T01:57:51.638689+0000","last_became_peered":"2024-04-23T01:57:51.638689+0000","last_unstale":"2024-04-23T01:57:56.675952+0000","last_undegraded":"2024-04-23T01:57:56.675952+0000","last_fullsized":"2024-04-23T01:57:56.675952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163840,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":200,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:52.522373+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:57:52.522373+0000","last_peered":"2024-04-23T01:57:52.522373+0000","last_clean":"2024-04-23T01:57:52.522373+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:57:52.522373+0000","last_undegraded":"2024-04-23T01:57:52.522373+0000","last_fullsized":"2024-04-23T01:57:52.522373+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":7651328,"num_objects":118,"num_object_clones":0,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":934,"num_write_kb":7472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":9920512,"data_stored":19841024,"data_compressed":1022028,"data_compressed_allocated":9920512,"data_compressed_original":19841024,"omap_allocated":0,"internal_metadata":0},"log_size":918,"ondisk_log_size":918,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706433,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":131716,"kb_used_data":2172,"kb_used_omap":0,"kb_used_meta":129536,"kb_avail":94240124,"statfs":{"total":96636764160,"available":96501886976,"internally_reserved":0,"allocated":2224128,"data_stored":3739701,"data_compressed":178437,"data_compressed_allocated":1482752,"data_compressed_original":3457024,"omap_allocated":0,"internal_metadata":132644864},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706433,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":110104,"kb_used_data":2768,"kb_used_omap":0,"kb_used_meta":107328,"kb_avail":94261736,"statfs":{"total":96636764160,"available":96524017664,"internally_reserved":0,"allocated":2834432,"data_stored":4960309,"data_compressed":241273,"data_compressed_allocated":2093056,"data_compressed_original":4677632,"omap_allocated":0,"internal_metadata":109903872},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706432,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":113704,"kb_used_data":2208,"kb_used_omap":0,"kb_used_meta":111488,"kb_avail":94258136,"statfs":{"total":96636764160,"available":96520331264,"internally_reserved":0,"allocated":2260992,"data_stored":3345941,"data_compressed":158578,"data_compressed_allocated":1540096,"data_compressed_original":3080192,"omap_allocated":0,"internal_metadata":114163712},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706437,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":130044,"kb_used_data":4660,"kb_used_omap":0,"kb_used_meta":125376,"kb_avail":94241796,"statfs":{"total":96636764160,"available":96503599104,"internally_reserved":0,"allocated":4771840,"data_stored":8835125,"data_compressed":441097,"data_compressed_allocated":4030464,"data_compressed_original":8552448,"omap_allocated":0,"internal_metadata":128385024},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4333568,"data_stored":8667136,"data_compressed":446676,"data_compressed_allocated":4333568,"data_compressed_original":8667136,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1937408,"data_stored":3874816,"data_compressed":199512,"data_compressed_allocated":1937408,"data_compressed_original":3874816,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2203648,"data_stored":4407296,"data_compressed":226934,"data_compressed_allocated":2203648,"data_compressed_original":4407296,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1445888,"data_stored":2891776,"data_compressed":148906,"data_compressed_allocated":1445888,"data_compressed_original":2891776,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:58:01.871 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-23T01:58:01.871 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:58:01.872 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:58:02.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:01.987+0000 7fd4fde6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:02.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:02.098+0000 7fd4fde6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:02.340 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:58:02.341 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:58:02.360 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2024-04-23T01:58:01.699823+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8241696,"num_objects":120,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":1071,"num_write_kb":9366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1015,"ondisk_log_size":1015,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":485568,"kb_used_data":11808,"kb_used_omap":0,"kb_used_meta":473728,"kb_avail":377001792,"statfs":{"total":386547056640,"available":386049835008,"internally_reserved":0,"allocated":12091392,"data_stored":20881076,"data_compressed":1019385,"data_compressed_allocated":9146368,"data_compressed_original":19767296,"omap_allocated":0,"internal_metadata":485097472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7651328,"num_objects":118,"num_object_clones":0,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":934,"num_write_kb":7472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001951"},"pg_stats":[{"pgid":"10.7","version":"84'71","reported_seq":84,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:58:00.559083+0000","last_change":"2024-04-23T01:57:51.583793+0000","last_active":"2024-04-23T01:58:00.559083+0000","last_peered":"2024-04-23T01:58:00.559083+0000","last_clean":"2024-04-23T01:58:00.559083+0000","last_became_active":"2024-04-23T01:57:51.574869+0000","last_became_peered":"2024-04-23T01:57:51.574869+0000","last_unstale":"2024-04-23T01:58:00.559083+0000","last_undegraded":"2024-04-23T01:58:00.559083+0000","last_fullsized":"2024-04-23T01:58:00.559083+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"84'31","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:54.561761+0000","last_change":"2024-04-23T01:57:51.679265+0000","last_active":"2024-04-23T01:57:54.561761+0000","last_peered":"2024-04-23T01:57:54.561761+0000","last_clean":"2024-04-23T01:57:54.561761+0000","last_became_active":"2024-04-23T01:57:51.662859+0000","last_became_peered":"2024-04-23T01:57:51.662859+0000","last_unstale":"2024-04-23T01:57:54.561761+0000","last_undegraded":"2024-04-23T01:57:54.561761+0000","last_fullsized":"2024-04-23T01:57:54.561761+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"84'57","reported_seq":70,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:56.727969+0000","last_change":"2024-04-23T01:57:51.593118+0000","last_active":"2024-04-23T01:57:56.727969+0000","last_peered":"2024-04-23T01:57:56.727969+0000","last_clean":"2024-04-23T01:57:56.727969+0000","last_became_active":"2024-04-23T01:57:51.580553+0000","last_became_peered":"2024-04-23T01:57:51.580553+0000","last_unstale":"2024-04-23T01:57:56.727969+0000","last_undegraded":"2024-04-23T01:57:56.727969+0000","last_fullsized":"2024-04-23T01:57:56.727969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":475136,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58,"num_write_kb":464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"84'15","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:56.231476+0000","last_change":"2024-04-23T01:57:51.577069+0000","last_active":"2024-04-23T01:57:56.231476+0000","last_peered":"2024-04-23T01:57:56.231476+0000","last_clean":"2024-04-23T01:57:56.231476+0000","last_became_active":"2024-04-23T01:57:51.568227+0000","last_became_peered":"2024-04-23T01:57:51.568227+0000","last_unstale":"2024-04-23T01:57:56.231476+0000","last_undegraded":"2024-04-23T01:57:56.231476+0000","last_fullsized":"2024-04-23T01:57:56.231476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"84'87","reported_seq":100,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:58:01.456659+0000","last_change":"2024-04-23T01:57:51.748592+0000","last_active":"2024-04-23T01:58:01.456659+0000","last_peered":"2024-04-23T01:58:01.456659+0000","last_clean":"2024-04-23T01:58:01.456659+0000","last_became_active":"2024-04-23T01:57:51.742159+0000","last_became_peered":"2024-04-23T01:57:51.742159+0000","last_unstale":"2024-04-23T01:58:01.456659+0000","last_undegraded":"2024-04-23T01:58:01.456659+0000","last_fullsized":"2024-04-23T01:58:01.456659+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":720896,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"84'39","reported_seq":52,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:56.525809+0000","last_change":"2024-04-23T01:57:51.630095+0000","last_active":"2024-04-23T01:57:56.525809+0000","last_peered":"2024-04-23T01:57:56.525809+0000","last_clean":"2024-04-23T01:57:56.525809+0000","last_became_active":"2024-04-23T01:57:51.623783+0000","last_became_peered":"2024-04-23T01:57:51.623783+0000","last_unstale":"2024-04-23T01:57:56.525809+0000","last_undegraded":"2024-04-23T01:57:56.525809+0000","last_fullsized":"2024-04-23T01:57:56.525809+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"84'63","reported_seq":76,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:58:01.242686+0000","last_change":"2024-04-23T01:57:51.605374+0000","last_active":"2024-04-23T01:58:01.242686+0000","last_peered":"2024-04-23T01:58:01.242686+0000","last_clean":"2024-04-23T01:58:01.242686+0000","last_became_active":"2024-04-23T01:57:51.597053+0000","last_became_peered":"2024-04-23T01:57:51.597053+0000","last_unstale":"2024-04-23T01:58:01.242686+0000","last_undegraded":"2024-04-23T01:58:01.242686+0000","last_fullsized":"2024-04-23T01:58:01.242686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"84'47","reported_seq":60,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:56.063329+0000","last_change":"2024-04-23T01:57:51.641687+0000","last_active":"2024-04-23T01:57:56.063329+0000","last_peered":"2024-04-23T01:57:56.063329+0000","last_clean":"2024-04-23T01:57:56.063329+0000","last_became_active":"2024-04-23T01:57:51.623973+0000","last_became_peered":"2024-04-23T01:57:51.623973+0000","last_unstale":"2024-04-23T01:57:56.063329+0000","last_undegraded":"2024-04-23T01:57:56.063329+0000","last_fullsized":"2024-04-23T01:57:56.063329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"84'47","reported_seq":60,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:56.597244+0000","last_change":"2024-04-23T01:57:51.669918+0000","last_active":"2024-04-23T01:57:56.597244+0000","last_peered":"2024-04-23T01:57:56.597244+0000","last_clean":"2024-04-23T01:57:56.597244+0000","last_became_active":"2024-04-23T01:57:51.652610+0000","last_became_peered":"2024-04-23T01:57:51.652610+0000","last_unstale":"2024-04-23T01:57:56.597244+0000","last_undegraded":"2024-04-23T01:57:56.597244+0000","last_fullsized":"2024-04-23T01:57:56.597244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":331,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:52.416961+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:57:52.416961+0000","last_peered":"2024-04-23T01:57:52.416961+0000","last_clean":"2024-04-23T01:57:52.416961+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:57:52.416961+0000","last_undegraded":"2024-04-23T01:57:52.416961+0000","last_fullsized":"2024-04-23T01:57:52.416961+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":200,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:52.527962+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:57:52.527962+0000","last_peered":"2024-04-23T01:57:52.527962+0000","last_clean":"2024-04-23T01:57:52.527962+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:57:52.527962+0000","last_undegraded":"2024-04-23T01:57:52.527962+0000","last_fullsized":"2024-04-23T01:57:52.527962+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"84'31","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:55.844213+0000","last_change":"2024-04-23T01:57:51.587471+0000","last_active":"2024-04-23T01:57:55.844213+0000","last_peered":"2024-04-23T01:57:55.844213+0000","last_clean":"2024-04-23T01:57:55.844213+0000","last_became_active":"2024-04-23T01:57:51.575352+0000","last_became_peered":"2024-04-23T01:57:51.575352+0000","last_unstale":"2024-04-23T01:57:55.844213+0000","last_undegraded":"2024-04-23T01:57:55.844213+0000","last_fullsized":"2024-04-23T01:57:55.844213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":192,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:52.589906+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:57:52.589906+0000","last_peered":"2024-04-23T01:57:52.589906+0000","last_clean":"2024-04-23T01:57:52.589906+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:57:52.589906+0000","last_undegraded":"2024-04-23T01:57:52.589906+0000","last_fullsized":"2024-04-23T01:57:52.589906+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"84'47","reported_seq":60,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:55.840253+0000","last_change":"2024-04-23T01:57:51.596134+0000","last_active":"2024-04-23T01:57:55.840253+0000","last_peered":"2024-04-23T01:57:55.840253+0000","last_clean":"2024-04-23T01:57:55.840253+0000","last_became_active":"2024-04-23T01:57:51.589605+0000","last_became_peered":"2024-04-23T01:57:51.589605+0000","last_unstale":"2024-04-23T01:57:55.840253+0000","last_undegraded":"2024-04-23T01:57:55.840253+0000","last_fullsized":"2024-04-23T01:57:55.840253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":200,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:52.519415+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:57:52.519415+0000","last_peered":"2024-04-23T01:57:52.519415+0000","last_clean":"2024-04-23T01:57:52.519415+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:57:52.519415+0000","last_undegraded":"2024-04-23T01:57:52.519415+0000","last_fullsized":"2024-04-23T01:57:52.519415+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"84'71","reported_seq":84,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:58:01.557809+0000","last_change":"2024-04-23T01:57:51.662237+0000","last_active":"2024-04-23T01:58:01.557809+0000","last_peered":"2024-04-23T01:58:01.557809+0000","last_clean":"2024-04-23T01:58:01.557809+0000","last_became_active":"2024-04-23T01:57:51.647583+0000","last_became_peered":"2024-04-23T01:57:51.647583+0000","last_unstale":"2024-04-23T01:58:01.557809+0000","last_undegraded":"2024-04-23T01:58:01.557809+0000","last_fullsized":"2024-04-23T01:58:01.557809+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":192,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:52.620913+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:57:52.620913+0000","last_peered":"2024-04-23T01:57:52.620913+0000","last_clean":"2024-04-23T01:57:52.620913+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:57:52.620913+0000","last_undegraded":"2024-04-23T01:57:52.620913+0000","last_fullsized":"2024-04-23T01:57:52.620913+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"84'79","reported_seq":92,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:58:01.071312+0000","last_change":"2024-04-23T01:57:51.681957+0000","last_active":"2024-04-23T01:58:01.071312+0000","last_peered":"2024-04-23T01:58:01.071312+0000","last_clean":"2024-04-23T01:58:01.071312+0000","last_became_active":"2024-04-23T01:57:51.670788+0000","last_became_peered":"2024-04-23T01:57:51.670788+0000","last_unstale":"2024-04-23T01:58:01.071312+0000","last_undegraded":"2024-04-23T01:58:01.071312+0000","last_fullsized":"2024-04-23T01:58:01.071312+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":198,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:53.340562+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:57:53.340562+0000","last_peered":"2024-04-23T01:57:53.340562+0000","last_clean":"2024-04-23T01:57:53.340562+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:57:53.340562+0000","last_undegraded":"2024-04-23T01:57:53.340562+0000","last_fullsized":"2024-04-23T01:57:53.340562+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"84'159","reported_seq":172,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:58:01.395407+0000","last_change":"2024-04-23T01:57:51.632423+0000","last_active":"2024-04-23T01:58:01.395407+0000","last_peered":"2024-04-23T01:58:01.395407+0000","last_clean":"2024-04-23T01:58:01.395407+0000","last_became_active":"2024-04-23T01:57:51.624354+0000","last_became_peered":"2024-04-23T01:57:51.624354+0000","last_unstale":"2024-04-23T01:58:01.395407+0000","last_undegraded":"2024-04-23T01:58:01.395407+0000","last_fullsized":"2024-04-23T01:58:01.395407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310720,"num_objects":20,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":160,"num_write_kb":1280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":198,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:52.442219+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:57:52.442219+0000","last_peered":"2024-04-23T01:57:52.442219+0000","last_clean":"2024-04-23T01:57:52.442219+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:57:52.442219+0000","last_undegraded":"2024-04-23T01:57:52.442219+0000","last_fullsized":"2024-04-23T01:57:52.442219+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"84'55","reported_seq":68,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:58:00.229556+0000","last_change":"2024-04-23T01:57:51.735746+0000","last_active":"2024-04-23T01:58:00.229556+0000","last_peered":"2024-04-23T01:58:00.229556+0000","last_clean":"2024-04-23T01:58:00.229556+0000","last_became_active":"2024-04-23T01:57:51.728785+0000","last_became_peered":"2024-04-23T01:57:51.728785+0000","last_unstale":"2024-04-23T01:58:00.229556+0000","last_undegraded":"2024-04-23T01:58:00.229556+0000","last_fullsized":"2024-04-23T01:58:00.229556+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458752,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":198,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:53.355136+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:57:53.355136+0000","last_peered":"2024-04-23T01:57:53.355136+0000","last_clean":"2024-04-23T01:57:53.355136+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:57:53.355136+0000","last_undegraded":"2024-04-23T01:57:53.355136+0000","last_fullsized":"2024-04-23T01:57:53.355136+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"84'19","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:56.675952+0000","last_change":"2024-04-23T01:57:51.656207+0000","last_active":"2024-04-23T01:57:56.675952+0000","last_peered":"2024-04-23T01:57:56.675952+0000","last_clean":"2024-04-23T01:57:56.675952+0000","last_became_active":"2024-04-23T01:57:51.638689+0000","last_became_peered":"2024-04-23T01:57:51.638689+0000","last_unstale":"2024-04-23T01:57:56.675952+0000","last_undegraded":"2024-04-23T01:57:56.675952+0000","last_fullsized":"2024-04-23T01:57:56.675952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163840,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":200,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-23T01:57:52.522373+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:57:52.522373+0000","last_peered":"2024-04-23T01:57:52.522373+0000","last_clean":"2024-04-23T01:57:52.522373+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:57:52.522373+0000","last_undegraded":"2024-04-23T01:57:52.522373+0000","last_fullsized":"2024-04-23T01:57:52.522373+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":7651328,"num_objects":118,"num_object_clones":0,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":934,"num_write_kb":7472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":9920512,"data_stored":19841024,"data_compressed":1022028,"data_compressed_allocated":9920512,"data_compressed_original":19841024,"omap_allocated":0,"internal_metadata":0},"log_size":918,"ondisk_log_size":918,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706433,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":131716,"kb_used_data":2172,"kb_used_omap":0,"kb_used_meta":129536,"kb_avail":94240124,"statfs":{"total":96636764160,"available":96501886976,"internally_reserved":0,"allocated":2224128,"data_stored":3739701,"data_compressed":178437,"data_compressed_allocated":1482752,"data_compressed_original":3457024,"omap_allocated":0,"internal_metadata":132644864},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706433,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":110104,"kb_used_data":2768,"kb_used_omap":0,"kb_used_meta":107328,"kb_avail":94261736,"statfs":{"total":96636764160,"available":96524017664,"internally_reserved":0,"allocated":2834432,"data_stored":4960309,"data_compressed":241273,"data_compressed_allocated":2093056,"data_compressed_original":4677632,"omap_allocated":0,"internal_metadata":109903872},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706432,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":113704,"kb_used_data":2208,"kb_used_omap":0,"kb_used_meta":111488,"kb_avail":94258136,"statfs":{"total":96636764160,"available":96520331264,"internally_reserved":0,"allocated":2260992,"data_stored":3345941,"data_compressed":158578,"data_compressed_allocated":1540096,"data_compressed_original":3080192,"omap_allocated":0,"internal_metadata":114163712},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706437,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":130044,"kb_used_data":4660,"kb_used_omap":0,"kb_used_meta":125376,"kb_avail":94241796,"statfs":{"total":96636764160,"available":96503599104,"internally_reserved":0,"allocated":4771840,"data_stored":8835125,"data_compressed":441097,"data_compressed_allocated":4030464,"data_compressed_original":8552448,"omap_allocated":0,"internal_metadata":128385024},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4333568,"data_stored":8667136,"data_compressed":446676,"data_compressed_allocated":4333568,"data_compressed_original":8667136,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1937408,"data_stored":3874816,"data_compressed":199512,"data_compressed_allocated":1937408,"data_compressed_original":3874816,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2203648,"data_stored":4407296,"data_compressed":226934,"data_compressed_allocated":2203648,"data_compressed_original":4407296,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1445888,"data_stored":2891776,"data_compressed":148906,"data_compressed_allocated":1445888,"data_compressed_original":2891776,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:58:02.481 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 10 16 1459 1443 1.12724 1.05469 0.0972646 0.110446 2024-04-23T01:58:03.378 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:58:03.481 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 11 16 1613 1597 1.13413 1.20312 0.110974 0.109652 2024-04-23T01:58:03.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:03.500+0000 7f60db8ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:03.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:03.505+0000 7f60db8ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:04.482 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 12 16 1768 1752 1.14052 1.21094 0.0330957 0.10899 2024-04-23T01:58:05.482 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 13 16 1930 1914 1.15013 1.26562 0.135906 0.108064 2024-04-23T01:58:05.695 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:58:05.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:05.804+0000 7f0873eed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:05.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:05.811+0000 7f0873eed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:06.482 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 14 16 2062 2046 1.14164 1.03125 0.157092 0.108992 2024-04-23T01:58:07.362 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:58:07.363 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:58:07.363 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:58:07.363 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:58:07.482 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 15 16 2221 2205 1.14833 1.24219 0.0811727 0.10851 2024-04-23T01:58:07.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:07.492+0000 7f885c19d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:07.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:07.507+0000 7f885c19d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:07.582 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:58:07.582 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:58:07.582 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:58:07.582 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:58:07.582 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:58:07.582 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:58:07.582 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:58:07.583 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:58:07.583 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:58:07.583 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:58:07.583 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:58:07.583 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:58:07.586 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:58:07.586 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:58:07.587 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:58:07.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:07.715+0000 7fdfbffff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:07.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:07.727+0000 7fdfbffff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:07.841 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:58:07.841 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:58:07.841 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:58:07.841 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:58:07.841 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:58:07.841 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:58:07.841 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:58:07.841 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:58:07.841 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:58:07.841 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:58:07.841 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:58:07.842 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:58:07.845 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:58:07.845 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:58:07.846 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:58:07.940 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:58:07.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:07.971+0000 7f5b1922d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:07.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:07.984+0000 7f5b1922d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:08.010 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:58:08.010 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:58:08.010 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:58:08.010 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:58:08.010 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:58:08.011 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:58:08.011 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:58:08.011 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:58:08.011 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:58:08.011 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:58:08.011 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:58:08.011 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:58:08.013 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:58:08.013 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:58:08.014 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:58:08.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:08.066+0000 7ff2986ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:08.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:08.074+0000 7ff2986ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:08.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:08.184+0000 7fed4c86d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:08.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:08.198+0000 7fed4c86d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:08.221 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:58:08.222 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:58:08.222 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:58:08.222 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:58:08.222 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:58:08.222 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:58:08.222 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:58:08.222 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:58:08.222 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:58:08.222 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:58:08.222 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:58:08.222 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:58:08.224 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:58:08.224 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:58:08.482 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 16 16 2371 2355 1.1498 1.17188 0.0526485 0.10827 2024-04-23T01:58:09.482 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 17 16 2517 2501 1.14925 1.14062 0.222576 0.108113 2024-04-23T01:58:10.264 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:58:10.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:10.408+0000 7f30c002d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:10.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:10.415+0000 7f30c002d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:10.482 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 18 16 2672 2656 1.15267 1.21094 0.0895457 0.108111 2024-04-23T01:58:11.482 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 19 16 2820 2804 1.15286 1.15625 0.0496011 0.107927 2024-04-23T01:58:12.483 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:58:12.481619+0000 min lat: 0.0138177 max lat: 0.388479 avg lat: 0.108272 2024-04-23T01:58:12.483 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:58:12.483 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 20 16 2961 2945 1.15029 1.10156 0.134231 0.108272 2024-04-23T01:58:12.657 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:58:12.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:12.771+0000 7f34b8dff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:12.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:12.778+0000 7f34b359e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:13.225 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:58:13.225 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-23T01:58:13.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:13.377+0000 7fe07fc06640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:13.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:13.389+0000 7fe07fc06640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:13.482 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 21 16 3093 3077 1.14461 1.03125 0.148364 0.108569 2024-04-23T01:58:14.451 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.0 to 1 (10000) 2024-04-23T01:58:14.468 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-23T01:58:14.482 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 22 16 3230 3214 1.14123 1.07031 0.0768687 0.109041 2024-04-23T01:58:14.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:14.575+0000 7f151066d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:14.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:14.588+0000 7f151066d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:15.017 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:58:15.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:15.165+0000 7f54f3d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:15.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:15.172+0000 7f54f3d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:15.452 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.1 to 1 (10000) 2024-04-23T01:58:15.472 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-23T01:58:15.482 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 23 16 3352 3336 1.13305 0.953125 0.0969848 0.109975 2024-04-23T01:58:15.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:15.641+0000 7fd32099d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:15.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:15.653+0000 7fd32099d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:16.461 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.2 to 1 (10000) 2024-04-23T01:58:16.479 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-23T01:58:16.482 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 24 16 3470 3454 1.12425 0.921875 0.0473134 0.110904 2024-04-23T01:58:16.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:16.628+0000 7fe5e179d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:16.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:16.643+0000 7fe5e179d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:17.417 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:58:17.467 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.3 to 1 (10000) 2024-04-23T01:58:17.483 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 25 16 3610 3594 1.12302 1.09375 0.10648 0.111017 2024-04-23T01:58:17.485 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-23T01:58:17.485 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:58:17.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:17.535+0000 7f21ea59d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:17.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:17.541+0000 7f21ea59d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:17.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:17.604+0000 7f0cf96ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:17.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:17.614+0000 7f0cf96ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:17.914 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:58:17.915 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:58:17.934 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":477,"stamp":"2024-04-23T01:58:17.703038+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29106720,"num_objects":440,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":3618,"num_write_kb":29742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3566,"ondisk_log_size":3566,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":519068,"kb_used_data":43900,"kb_used_omap":0,"kb_used_meta":475136,"kb_avail":376968292,"statfs":{"total":386547056640,"available":386015531008,"internally_reserved":0,"allocated":44953600,"data_stored":86487241,"data_compressed":4404513,"data_compressed_allocated":41934848,"data_compressed_original":85344256,"omap_allocated":0,"internal_metadata":486539264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5242880,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":640,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000845"},"pg_stats":[{"pgid":"10.7","version":"86'247","reported_seq":266,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.316183+0000","last_change":"2024-04-23T01:57:51.583793+0000","last_active":"2024-04-23T01:58:16.316183+0000","last_peered":"2024-04-23T01:58:16.316183+0000","last_clean":"2024-04-23T01:58:16.316183+0000","last_became_active":"2024-04-23T01:57:51.574869+0000","last_became_peered":"2024-04-23T01:57:51.574869+0000","last_unstale":"2024-04-23T01:58:16.316183+0000","last_undegraded":"2024-04-23T01:58:16.316183+0000","last_fullsized":"2024-04-23T01:58:16.316183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"86'285","reported_seq":304,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.657609+0000","last_change":"2024-04-23T01:57:51.679265+0000","last_active":"2024-04-23T01:58:16.657609+0000","last_peered":"2024-04-23T01:58:16.657609+0000","last_clean":"2024-04-23T01:58:16.657609+0000","last_became_active":"2024-04-23T01:57:51.662859+0000","last_became_peered":"2024-04-23T01:57:51.662859+0000","last_unstale":"2024-04-23T01:58:16.657609+0000","last_undegraded":"2024-04-23T01:58:16.657609+0000","last_fullsized":"2024-04-23T01:58:16.657609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2342912,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":286,"num_write_kb":2288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"84'232","reported_seq":250,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.602770+0000","last_change":"2024-04-23T01:57:51.593118+0000","last_active":"2024-04-23T01:58:15.602770+0000","last_peered":"2024-04-23T01:58:15.602770+0000","last_clean":"2024-04-23T01:58:15.602770+0000","last_became_active":"2024-04-23T01:57:51.580553+0000","last_became_peered":"2024-04-23T01:57:51.580553+0000","last_unstale":"2024-04-23T01:58:15.602770+0000","last_undegraded":"2024-04-23T01:58:15.602770+0000","last_fullsized":"2024-04-23T01:58:15.602770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1900544,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":232,"num_write_kb":1856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"84'184","reported_seq":205,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.496287+0000","last_change":"2024-04-23T01:57:51.577069+0000","last_active":"2024-04-23T01:58:16.496287+0000","last_peered":"2024-04-23T01:58:16.496287+0000","last_clean":"2024-04-23T01:58:16.496287+0000","last_became_active":"2024-04-23T01:57:51.568227+0000","last_became_peered":"2024-04-23T01:57:51.568227+0000","last_unstale":"2024-04-23T01:58:16.496287+0000","last_undegraded":"2024-04-23T01:58:16.496287+0000","last_fullsized":"2024-04-23T01:58:16.496287+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"86'223","reported_seq":242,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.565428+0000","last_change":"2024-04-23T01:57:51.748592+0000","last_active":"2024-04-23T01:58:15.565428+0000","last_peered":"2024-04-23T01:58:15.565428+0000","last_clean":"2024-04-23T01:58:15.565428+0000","last_became_active":"2024-04-23T01:57:51.742159+0000","last_became_peered":"2024-04-23T01:57:51.742159+0000","last_unstale":"2024-04-23T01:58:15.565428+0000","last_undegraded":"2024-04-23T01:58:15.565428+0000","last_fullsized":"2024-04-23T01:58:15.565428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1835008,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":1792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"86'216","reported_seq":238,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.507137+0000","last_change":"2024-04-23T01:57:51.630095+0000","last_active":"2024-04-23T01:58:16.507137+0000","last_peered":"2024-04-23T01:58:16.507137+0000","last_clean":"2024-04-23T01:58:16.507137+0000","last_became_active":"2024-04-23T01:57:51.623783+0000","last_became_peered":"2024-04-23T01:57:51.623783+0000","last_unstale":"2024-04-23T01:58:16.507137+0000","last_undegraded":"2024-04-23T01:58:16.507137+0000","last_fullsized":"2024-04-23T01:58:16.507137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1777664,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":217,"num_write_kb":1736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"86'144","reported_seq":163,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.215058+0000","last_change":"2024-04-23T01:57:51.605374+0000","last_active":"2024-04-23T01:58:16.215058+0000","last_peered":"2024-04-23T01:58:16.215058+0000","last_clean":"2024-04-23T01:58:16.215058+0000","last_became_active":"2024-04-23T01:57:51.597053+0000","last_became_peered":"2024-04-23T01:57:51.597053+0000","last_unstale":"2024-04-23T01:58:16.215058+0000","last_undegraded":"2024-04-23T01:58:16.215058+0000","last_fullsized":"2024-04-23T01:58:16.215058+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1187840,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":145,"num_write_kb":1160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"86'287","reported_seq":306,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.026088+0000","last_change":"2024-04-23T01:57:51.641687+0000","last_active":"2024-04-23T01:58:16.026088+0000","last_peered":"2024-04-23T01:58:16.026088+0000","last_clean":"2024-04-23T01:58:16.026088+0000","last_became_active":"2024-04-23T01:57:51.623973+0000","last_became_peered":"2024-04-23T01:57:51.623973+0000","last_unstale":"2024-04-23T01:58:16.026088+0000","last_undegraded":"2024-04-23T01:58:16.026088+0000","last_fullsized":"2024-04-23T01:58:16.026088+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"84'184","reported_seq":202,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.657684+0000","last_change":"2024-04-23T01:57:51.669918+0000","last_active":"2024-04-23T01:58:15.657684+0000","last_peered":"2024-04-23T01:58:15.657684+0000","last_clean":"2024-04-23T01:58:15.657684+0000","last_became_active":"2024-04-23T01:57:51.652610+0000","last_became_peered":"2024-04-23T01:57:51.652610+0000","last_unstale":"2024-04-23T01:58:15.657684+0000","last_undegraded":"2024-04-23T01:58:15.657684+0000","last_fullsized":"2024-04-23T01:58:15.657684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":340,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.486220+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:58:16.486220+0000","last_peered":"2024-04-23T01:58:16.486220+0000","last_clean":"2024-04-23T01:58:16.486220+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:58:16.486220+0000","last_undegraded":"2024-04-23T01:58:16.486220+0000","last_fullsized":"2024-04-23T01:58:16.486220+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":206,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.674360+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:58:15.674360+0000","last_peered":"2024-04-23T01:58:15.674360+0000","last_clean":"2024-04-23T01:58:15.674360+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:58:15.674360+0000","last_undegraded":"2024-04-23T01:58:15.674360+0000","last_fullsized":"2024-04-23T01:58:15.674360+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"86'227","reported_seq":246,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.746931+0000","last_change":"2024-04-23T01:57:51.587471+0000","last_active":"2024-04-23T01:58:16.746931+0000","last_peered":"2024-04-23T01:58:16.746931+0000","last_clean":"2024-04-23T01:58:16.746931+0000","last_became_active":"2024-04-23T01:57:51.575352+0000","last_became_peered":"2024-04-23T01:57:51.575352+0000","last_unstale":"2024-04-23T01:58:16.746931+0000","last_undegraded":"2024-04-23T01:58:16.746931+0000","last_fullsized":"2024-04-23T01:58:16.746931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1867776,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":228,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":198,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.511664+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:58:15.511664+0000","last_peered":"2024-04-23T01:58:15.511664+0000","last_clean":"2024-04-23T01:58:15.511664+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:58:15.511664+0000","last_undegraded":"2024-04-23T01:58:15.511664+0000","last_fullsized":"2024-04-23T01:58:15.511664+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"87'271","reported_seq":293,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.572887+0000","last_change":"2024-04-23T01:57:51.596134+0000","last_active":"2024-04-23T01:58:16.572887+0000","last_peered":"2024-04-23T01:58:16.572887+0000","last_clean":"2024-04-23T01:58:16.572887+0000","last_became_active":"2024-04-23T01:57:51.589605+0000","last_became_peered":"2024-04-23T01:57:51.589605+0000","last_unstale":"2024-04-23T01:58:16.572887+0000","last_undegraded":"2024-04-23T01:58:16.572887+0000","last_fullsized":"2024-04-23T01:58:16.572887+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":272,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":206,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.659245+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:58:15.659245+0000","last_peered":"2024-04-23T01:58:15.659245+0000","last_clean":"2024-04-23T01:58:15.659245+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:58:15.659245+0000","last_undegraded":"2024-04-23T01:58:15.659245+0000","last_fullsized":"2024-04-23T01:58:15.659245+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"84'176","reported_seq":194,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.487735+0000","last_change":"2024-04-23T01:57:51.662237+0000","last_active":"2024-04-23T01:58:15.487735+0000","last_peered":"2024-04-23T01:58:15.487735+0000","last_clean":"2024-04-23T01:58:15.487735+0000","last_became_active":"2024-04-23T01:57:51.647583+0000","last_became_peered":"2024-04-23T01:57:51.647583+0000","last_unstale":"2024-04-23T01:58:15.487735+0000","last_undegraded":"2024-04-23T01:58:15.487735+0000","last_fullsized":"2024-04-23T01:58:15.487735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":198,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.508558+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:58:15.508558+0000","last_peered":"2024-04-23T01:58:15.508558+0000","last_clean":"2024-04-23T01:58:15.508558+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:58:15.508558+0000","last_undegraded":"2024-04-23T01:58:15.508558+0000","last_fullsized":"2024-04-23T01:58:15.508558+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"86'199","reported_seq":218,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.524373+0000","last_change":"2024-04-23T01:57:51.681957+0000","last_active":"2024-04-23T01:58:16.524373+0000","last_peered":"2024-04-23T01:58:16.524373+0000","last_clean":"2024-04-23T01:58:16.524373+0000","last_became_active":"2024-04-23T01:57:51.670788+0000","last_became_peered":"2024-04-23T01:57:51.670788+0000","last_unstale":"2024-04-23T01:58:16.524373+0000","last_undegraded":"2024-04-23T01:58:16.524373+0000","last_fullsized":"2024-04-23T01:58:16.524373+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":200,"num_write_kb":1600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":204,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.613866+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:58:15.613866+0000","last_peered":"2024-04-23T01:58:15.613866+0000","last_clean":"2024-04-23T01:58:15.613866+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:58:15.613866+0000","last_undegraded":"2024-04-23T01:58:15.613866+0000","last_fullsized":"2024-04-23T01:58:15.613866+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"86'260","reported_seq":279,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.644099+0000","last_change":"2024-04-23T01:57:51.632423+0000","last_active":"2024-04-23T01:58:16.644099+0000","last_peered":"2024-04-23T01:58:16.644099+0000","last_clean":"2024-04-23T01:58:16.644099+0000","last_became_active":"2024-04-23T01:57:51.624354+0000","last_became_peered":"2024-04-23T01:57:51.624354+0000","last_unstale":"2024-04-23T01:58:16.644099+0000","last_undegraded":"2024-04-23T01:58:16.644099+0000","last_fullsized":"2024-04-23T01:58:16.644099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2138112,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":261,"num_write_kb":2088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":207,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.499984+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:58:16.499984+0000","last_peered":"2024-04-23T01:58:16.499984+0000","last_clean":"2024-04-23T01:58:16.499984+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:58:16.499984+0000","last_undegraded":"2024-04-23T01:58:16.499984+0000","last_fullsized":"2024-04-23T01:58:16.499984+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"86'183","reported_seq":202,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.410714+0000","last_change":"2024-04-23T01:57:51.735746+0000","last_active":"2024-04-23T01:58:16.410714+0000","last_peered":"2024-04-23T01:58:16.410714+0000","last_clean":"2024-04-23T01:58:16.410714+0000","last_became_active":"2024-04-23T01:57:51.728785+0000","last_became_peered":"2024-04-23T01:57:51.728785+0000","last_unstale":"2024-04-23T01:58:16.410714+0000","last_undegraded":"2024-04-23T01:58:16.410714+0000","last_fullsized":"2024-04-23T01:58:16.410714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":204,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.609718+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:58:15.609718+0000","last_peered":"2024-04-23T01:58:15.609718+0000","last_clean":"2024-04-23T01:58:15.609718+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:58:15.609718+0000","last_undegraded":"2024-04-23T01:58:15.609718+0000","last_fullsized":"2024-04-23T01:58:15.609718+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"86'151","reported_seq":170,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.711529+0000","last_change":"2024-04-23T01:57:51.656207+0000","last_active":"2024-04-23T01:58:15.711529+0000","last_peered":"2024-04-23T01:58:15.711529+0000","last_clean":"2024-04-23T01:58:15.711529+0000","last_became_active":"2024-04-23T01:57:51.638689+0000","last_became_peered":"2024-04-23T01:57:51.638689+0000","last_unstale":"2024-04-23T01:58:15.711529+0000","last_undegraded":"2024-04-23T01:58:15.711529+0000","last_fullsized":"2024-04-23T01:58:15.711529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":206,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.661790+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:58:15.661790+0000","last_peered":"2024-04-23T01:58:15.661790+0000","last_clean":"2024-04-23T01:58:15.661790+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:58:15.661790+0000","last_undegraded":"2024-04-23T01:58:15.661790+0000","last_fullsized":"2024-04-23T01:58:15.661790+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":28516352,"num_objects":438,"num_object_clones":0,"num_object_copies":1314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3481,"num_write_kb":27848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":42786816,"data_stored":85573632,"data_compressed":4415328,"data_compressed_allocated":42786816,"data_compressed_original":85573632,"omap_allocated":0,"internal_metadata":0},"log_size":3469,"ondisk_log_size":3469,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706453,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":129952,"kb_used_data":8728,"kb_used_omap":0,"kb_used_meta":121216,"kb_avail":94241888,"statfs":{"total":96636764160,"available":96503693312,"internally_reserved":0,"allocated":8937472,"data_stored":17126972,"data_compressed":869012,"data_compressed_allocated":8171520,"data_compressed_original":16834560,"omap_allocated":0,"internal_metadata":124125184},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706453,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":115856,"kb_used_data":12680,"kb_used_omap":0,"kb_used_meta":103168,"kb_avail":94255984,"statfs":{"total":96636764160,"available":96518127616,"internally_reserved":0,"allocated":12984320,"data_stored":25233807,"data_compressed":1287388,"data_compressed_allocated":12226560,"data_compressed_original":24944640,"omap_allocated":0,"internal_metadata":105644032},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706452,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":131860,"kb_used_data":10636,"kb_used_omap":0,"kb_used_meta":121216,"kb_avail":94239980,"statfs":{"total":96636764160,"available":96501739520,"internally_reserved":0,"allocated":10891264,"data_stored":20580207,"data_compressed":1047731,"data_compressed_allocated":10153984,"data_compressed_original":20307968,"omap_allocated":0,"internal_metadata":124125184},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706452,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":141400,"kb_used_data":11856,"kb_used_omap":0,"kb_used_meta":129536,"kb_avail":94230440,"statfs":{"total":96636764160,"available":96491970560,"internally_reserved":0,"allocated":12140544,"data_stored":23546255,"data_compressed":1200382,"data_compressed_allocated":11382784,"data_compressed_original":23257088,"omap_allocated":0,"internal_metadata":132644864},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":11776000,"data_stored":23552000,"data_compressed":1215267,"data_compressed_allocated":11776000,"data_compressed_original":23552000,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10575872,"data_stored":21151744,"data_compressed":1091300,"data_compressed_allocated":10575872,"data_compressed_original":21151744,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12292096,"data_stored":24584192,"data_compressed":1268433,"data_compressed_allocated":12292096,"data_compressed_original":24584192,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8142848,"data_stored":16285696,"data_compressed":840328,"data_compressed_allocated":8142848,"data_compressed_original":16285696,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:58:17.935 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:58:18.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:18.069+0000 7f91af39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:18.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:18.079+0000 7f91af39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:18.286 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:58:18.286 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:58:18.307 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":477,"stamp":"2024-04-23T01:58:17.703038+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29106720,"num_objects":440,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":3618,"num_write_kb":29742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3566,"ondisk_log_size":3566,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":519068,"kb_used_data":43900,"kb_used_omap":0,"kb_used_meta":475136,"kb_avail":376968292,"statfs":{"total":386547056640,"available":386015531008,"internally_reserved":0,"allocated":44953600,"data_stored":86487241,"data_compressed":4404513,"data_compressed_allocated":41934848,"data_compressed_original":85344256,"omap_allocated":0,"internal_metadata":486539264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5242880,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":640,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000845"},"pg_stats":[{"pgid":"10.7","version":"86'247","reported_seq":266,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.316183+0000","last_change":"2024-04-23T01:57:51.583793+0000","last_active":"2024-04-23T01:58:16.316183+0000","last_peered":"2024-04-23T01:58:16.316183+0000","last_clean":"2024-04-23T01:58:16.316183+0000","last_became_active":"2024-04-23T01:57:51.574869+0000","last_became_peered":"2024-04-23T01:57:51.574869+0000","last_unstale":"2024-04-23T01:58:16.316183+0000","last_undegraded":"2024-04-23T01:58:16.316183+0000","last_fullsized":"2024-04-23T01:58:16.316183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"86'285","reported_seq":304,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.657609+0000","last_change":"2024-04-23T01:57:51.679265+0000","last_active":"2024-04-23T01:58:16.657609+0000","last_peered":"2024-04-23T01:58:16.657609+0000","last_clean":"2024-04-23T01:58:16.657609+0000","last_became_active":"2024-04-23T01:57:51.662859+0000","last_became_peered":"2024-04-23T01:57:51.662859+0000","last_unstale":"2024-04-23T01:58:16.657609+0000","last_undegraded":"2024-04-23T01:58:16.657609+0000","last_fullsized":"2024-04-23T01:58:16.657609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2342912,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":286,"num_write_kb":2288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"84'232","reported_seq":250,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.602770+0000","last_change":"2024-04-23T01:57:51.593118+0000","last_active":"2024-04-23T01:58:15.602770+0000","last_peered":"2024-04-23T01:58:15.602770+0000","last_clean":"2024-04-23T01:58:15.602770+0000","last_became_active":"2024-04-23T01:57:51.580553+0000","last_became_peered":"2024-04-23T01:57:51.580553+0000","last_unstale":"2024-04-23T01:58:15.602770+0000","last_undegraded":"2024-04-23T01:58:15.602770+0000","last_fullsized":"2024-04-23T01:58:15.602770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1900544,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":232,"num_write_kb":1856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"84'184","reported_seq":205,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.496287+0000","last_change":"2024-04-23T01:57:51.577069+0000","last_active":"2024-04-23T01:58:16.496287+0000","last_peered":"2024-04-23T01:58:16.496287+0000","last_clean":"2024-04-23T01:58:16.496287+0000","last_became_active":"2024-04-23T01:57:51.568227+0000","last_became_peered":"2024-04-23T01:57:51.568227+0000","last_unstale":"2024-04-23T01:58:16.496287+0000","last_undegraded":"2024-04-23T01:58:16.496287+0000","last_fullsized":"2024-04-23T01:58:16.496287+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"86'223","reported_seq":242,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.565428+0000","last_change":"2024-04-23T01:57:51.748592+0000","last_active":"2024-04-23T01:58:15.565428+0000","last_peered":"2024-04-23T01:58:15.565428+0000","last_clean":"2024-04-23T01:58:15.565428+0000","last_became_active":"2024-04-23T01:57:51.742159+0000","last_became_peered":"2024-04-23T01:57:51.742159+0000","last_unstale":"2024-04-23T01:58:15.565428+0000","last_undegraded":"2024-04-23T01:58:15.565428+0000","last_fullsized":"2024-04-23T01:58:15.565428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1835008,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":1792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"86'216","reported_seq":238,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.507137+0000","last_change":"2024-04-23T01:57:51.630095+0000","last_active":"2024-04-23T01:58:16.507137+0000","last_peered":"2024-04-23T01:58:16.507137+0000","last_clean":"2024-04-23T01:58:16.507137+0000","last_became_active":"2024-04-23T01:57:51.623783+0000","last_became_peered":"2024-04-23T01:57:51.623783+0000","last_unstale":"2024-04-23T01:58:16.507137+0000","last_undegraded":"2024-04-23T01:58:16.507137+0000","last_fullsized":"2024-04-23T01:58:16.507137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1777664,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":217,"num_write_kb":1736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"86'144","reported_seq":163,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.215058+0000","last_change":"2024-04-23T01:57:51.605374+0000","last_active":"2024-04-23T01:58:16.215058+0000","last_peered":"2024-04-23T01:58:16.215058+0000","last_clean":"2024-04-23T01:58:16.215058+0000","last_became_active":"2024-04-23T01:57:51.597053+0000","last_became_peered":"2024-04-23T01:57:51.597053+0000","last_unstale":"2024-04-23T01:58:16.215058+0000","last_undegraded":"2024-04-23T01:58:16.215058+0000","last_fullsized":"2024-04-23T01:58:16.215058+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1187840,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":145,"num_write_kb":1160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"86'287","reported_seq":306,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.026088+0000","last_change":"2024-04-23T01:57:51.641687+0000","last_active":"2024-04-23T01:58:16.026088+0000","last_peered":"2024-04-23T01:58:16.026088+0000","last_clean":"2024-04-23T01:58:16.026088+0000","last_became_active":"2024-04-23T01:57:51.623973+0000","last_became_peered":"2024-04-23T01:57:51.623973+0000","last_unstale":"2024-04-23T01:58:16.026088+0000","last_undegraded":"2024-04-23T01:58:16.026088+0000","last_fullsized":"2024-04-23T01:58:16.026088+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"84'184","reported_seq":202,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.657684+0000","last_change":"2024-04-23T01:57:51.669918+0000","last_active":"2024-04-23T01:58:15.657684+0000","last_peered":"2024-04-23T01:58:15.657684+0000","last_clean":"2024-04-23T01:58:15.657684+0000","last_became_active":"2024-04-23T01:57:51.652610+0000","last_became_peered":"2024-04-23T01:57:51.652610+0000","last_unstale":"2024-04-23T01:58:15.657684+0000","last_undegraded":"2024-04-23T01:58:15.657684+0000","last_fullsized":"2024-04-23T01:58:15.657684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":340,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.486220+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:58:16.486220+0000","last_peered":"2024-04-23T01:58:16.486220+0000","last_clean":"2024-04-23T01:58:16.486220+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:58:16.486220+0000","last_undegraded":"2024-04-23T01:58:16.486220+0000","last_fullsized":"2024-04-23T01:58:16.486220+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":206,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.674360+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:58:15.674360+0000","last_peered":"2024-04-23T01:58:15.674360+0000","last_clean":"2024-04-23T01:58:15.674360+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:58:15.674360+0000","last_undegraded":"2024-04-23T01:58:15.674360+0000","last_fullsized":"2024-04-23T01:58:15.674360+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"86'227","reported_seq":246,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.746931+0000","last_change":"2024-04-23T01:57:51.587471+0000","last_active":"2024-04-23T01:58:16.746931+0000","last_peered":"2024-04-23T01:58:16.746931+0000","last_clean":"2024-04-23T01:58:16.746931+0000","last_became_active":"2024-04-23T01:57:51.575352+0000","last_became_peered":"2024-04-23T01:57:51.575352+0000","last_unstale":"2024-04-23T01:58:16.746931+0000","last_undegraded":"2024-04-23T01:58:16.746931+0000","last_fullsized":"2024-04-23T01:58:16.746931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1867776,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":228,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":198,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.511664+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:58:15.511664+0000","last_peered":"2024-04-23T01:58:15.511664+0000","last_clean":"2024-04-23T01:58:15.511664+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:58:15.511664+0000","last_undegraded":"2024-04-23T01:58:15.511664+0000","last_fullsized":"2024-04-23T01:58:15.511664+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"87'271","reported_seq":293,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.572887+0000","last_change":"2024-04-23T01:57:51.596134+0000","last_active":"2024-04-23T01:58:16.572887+0000","last_peered":"2024-04-23T01:58:16.572887+0000","last_clean":"2024-04-23T01:58:16.572887+0000","last_became_active":"2024-04-23T01:57:51.589605+0000","last_became_peered":"2024-04-23T01:57:51.589605+0000","last_unstale":"2024-04-23T01:58:16.572887+0000","last_undegraded":"2024-04-23T01:58:16.572887+0000","last_fullsized":"2024-04-23T01:58:16.572887+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":272,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":206,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.659245+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:58:15.659245+0000","last_peered":"2024-04-23T01:58:15.659245+0000","last_clean":"2024-04-23T01:58:15.659245+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:58:15.659245+0000","last_undegraded":"2024-04-23T01:58:15.659245+0000","last_fullsized":"2024-04-23T01:58:15.659245+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"84'176","reported_seq":194,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.487735+0000","last_change":"2024-04-23T01:57:51.662237+0000","last_active":"2024-04-23T01:58:15.487735+0000","last_peered":"2024-04-23T01:58:15.487735+0000","last_clean":"2024-04-23T01:58:15.487735+0000","last_became_active":"2024-04-23T01:57:51.647583+0000","last_became_peered":"2024-04-23T01:57:51.647583+0000","last_unstale":"2024-04-23T01:58:15.487735+0000","last_undegraded":"2024-04-23T01:58:15.487735+0000","last_fullsized":"2024-04-23T01:58:15.487735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":198,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.508558+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:58:15.508558+0000","last_peered":"2024-04-23T01:58:15.508558+0000","last_clean":"2024-04-23T01:58:15.508558+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:58:15.508558+0000","last_undegraded":"2024-04-23T01:58:15.508558+0000","last_fullsized":"2024-04-23T01:58:15.508558+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"86'199","reported_seq":218,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.524373+0000","last_change":"2024-04-23T01:57:51.681957+0000","last_active":"2024-04-23T01:58:16.524373+0000","last_peered":"2024-04-23T01:58:16.524373+0000","last_clean":"2024-04-23T01:58:16.524373+0000","last_became_active":"2024-04-23T01:57:51.670788+0000","last_became_peered":"2024-04-23T01:57:51.670788+0000","last_unstale":"2024-04-23T01:58:16.524373+0000","last_undegraded":"2024-04-23T01:58:16.524373+0000","last_fullsized":"2024-04-23T01:58:16.524373+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":200,"num_write_kb":1600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":204,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.613866+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:58:15.613866+0000","last_peered":"2024-04-23T01:58:15.613866+0000","last_clean":"2024-04-23T01:58:15.613866+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:58:15.613866+0000","last_undegraded":"2024-04-23T01:58:15.613866+0000","last_fullsized":"2024-04-23T01:58:15.613866+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"86'260","reported_seq":279,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.644099+0000","last_change":"2024-04-23T01:57:51.632423+0000","last_active":"2024-04-23T01:58:16.644099+0000","last_peered":"2024-04-23T01:58:16.644099+0000","last_clean":"2024-04-23T01:58:16.644099+0000","last_became_active":"2024-04-23T01:57:51.624354+0000","last_became_peered":"2024-04-23T01:57:51.624354+0000","last_unstale":"2024-04-23T01:58:16.644099+0000","last_undegraded":"2024-04-23T01:58:16.644099+0000","last_fullsized":"2024-04-23T01:58:16.644099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2138112,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":261,"num_write_kb":2088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":207,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.499984+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:58:16.499984+0000","last_peered":"2024-04-23T01:58:16.499984+0000","last_clean":"2024-04-23T01:58:16.499984+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:58:16.499984+0000","last_undegraded":"2024-04-23T01:58:16.499984+0000","last_fullsized":"2024-04-23T01:58:16.499984+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"86'183","reported_seq":202,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.410714+0000","last_change":"2024-04-23T01:57:51.735746+0000","last_active":"2024-04-23T01:58:16.410714+0000","last_peered":"2024-04-23T01:58:16.410714+0000","last_clean":"2024-04-23T01:58:16.410714+0000","last_became_active":"2024-04-23T01:57:51.728785+0000","last_became_peered":"2024-04-23T01:57:51.728785+0000","last_unstale":"2024-04-23T01:58:16.410714+0000","last_undegraded":"2024-04-23T01:58:16.410714+0000","last_fullsized":"2024-04-23T01:58:16.410714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":204,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.609718+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:58:15.609718+0000","last_peered":"2024-04-23T01:58:15.609718+0000","last_clean":"2024-04-23T01:58:15.609718+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:58:15.609718+0000","last_undegraded":"2024-04-23T01:58:15.609718+0000","last_fullsized":"2024-04-23T01:58:15.609718+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"86'151","reported_seq":170,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.711529+0000","last_change":"2024-04-23T01:57:51.656207+0000","last_active":"2024-04-23T01:58:15.711529+0000","last_peered":"2024-04-23T01:58:15.711529+0000","last_clean":"2024-04-23T01:58:15.711529+0000","last_became_active":"2024-04-23T01:57:51.638689+0000","last_became_peered":"2024-04-23T01:57:51.638689+0000","last_unstale":"2024-04-23T01:58:15.711529+0000","last_undegraded":"2024-04-23T01:58:15.711529+0000","last_fullsized":"2024-04-23T01:58:15.711529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":206,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.661790+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:58:15.661790+0000","last_peered":"2024-04-23T01:58:15.661790+0000","last_clean":"2024-04-23T01:58:15.661790+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:58:15.661790+0000","last_undegraded":"2024-04-23T01:58:15.661790+0000","last_fullsized":"2024-04-23T01:58:15.661790+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":28516352,"num_objects":438,"num_object_clones":0,"num_object_copies":1314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3481,"num_write_kb":27848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":42786816,"data_stored":85573632,"data_compressed":4415328,"data_compressed_allocated":42786816,"data_compressed_original":85573632,"omap_allocated":0,"internal_metadata":0},"log_size":3469,"ondisk_log_size":3469,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706453,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":129952,"kb_used_data":8728,"kb_used_omap":0,"kb_used_meta":121216,"kb_avail":94241888,"statfs":{"total":96636764160,"available":96503693312,"internally_reserved":0,"allocated":8937472,"data_stored":17126972,"data_compressed":869012,"data_compressed_allocated":8171520,"data_compressed_original":16834560,"omap_allocated":0,"internal_metadata":124125184},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706453,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":115856,"kb_used_data":12680,"kb_used_omap":0,"kb_used_meta":103168,"kb_avail":94255984,"statfs":{"total":96636764160,"available":96518127616,"internally_reserved":0,"allocated":12984320,"data_stored":25233807,"data_compressed":1287388,"data_compressed_allocated":12226560,"data_compressed_original":24944640,"omap_allocated":0,"internal_metadata":105644032},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706452,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":131860,"kb_used_data":10636,"kb_used_omap":0,"kb_used_meta":121216,"kb_avail":94239980,"statfs":{"total":96636764160,"available":96501739520,"internally_reserved":0,"allocated":10891264,"data_stored":20580207,"data_compressed":1047731,"data_compressed_allocated":10153984,"data_compressed_original":20307968,"omap_allocated":0,"internal_metadata":124125184},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706452,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":141400,"kb_used_data":11856,"kb_used_omap":0,"kb_used_meta":129536,"kb_avail":94230440,"statfs":{"total":96636764160,"available":96491970560,"internally_reserved":0,"allocated":12140544,"data_stored":23546255,"data_compressed":1200382,"data_compressed_allocated":11382784,"data_compressed_original":23257088,"omap_allocated":0,"internal_metadata":132644864},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":11776000,"data_stored":23552000,"data_compressed":1215267,"data_compressed_allocated":11776000,"data_compressed_original":23552000,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10575872,"data_stored":21151744,"data_compressed":1091300,"data_compressed_allocated":10575872,"data_compressed_original":21151744,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12292096,"data_stored":24584192,"data_compressed":1268433,"data_compressed_allocated":12292096,"data_compressed_original":24584192,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8142848,"data_stored":16285696,"data_compressed":840328,"data_compressed_allocated":8142848,"data_compressed_original":16285696,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:58:18.308 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-23T01:58:18.309 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:58:18.309 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:58:18.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:18.430+0000 7f0a40aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:18.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:18.441+0000 7f0a40aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:18.483 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 26 16 3737 3721 1.11799 0.992188 0.169204 0.111554 2024-04-23T01:58:18.653 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:58:18.653 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:58:18.667 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":477,"stamp":"2024-04-23T01:58:17.703038+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29106720,"num_objects":440,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":3618,"num_write_kb":29742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3566,"ondisk_log_size":3566,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":519068,"kb_used_data":43900,"kb_used_omap":0,"kb_used_meta":475136,"kb_avail":376968292,"statfs":{"total":386547056640,"available":386015531008,"internally_reserved":0,"allocated":44953600,"data_stored":86487241,"data_compressed":4404513,"data_compressed_allocated":41934848,"data_compressed_original":85344256,"omap_allocated":0,"internal_metadata":486539264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5242880,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":640,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000845"},"pg_stats":[{"pgid":"10.7","version":"86'247","reported_seq":266,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.316183+0000","last_change":"2024-04-23T01:57:51.583793+0000","last_active":"2024-04-23T01:58:16.316183+0000","last_peered":"2024-04-23T01:58:16.316183+0000","last_clean":"2024-04-23T01:58:16.316183+0000","last_became_active":"2024-04-23T01:57:51.574869+0000","last_became_peered":"2024-04-23T01:57:51.574869+0000","last_unstale":"2024-04-23T01:58:16.316183+0000","last_undegraded":"2024-04-23T01:58:16.316183+0000","last_fullsized":"2024-04-23T01:58:16.316183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"86'285","reported_seq":304,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.657609+0000","last_change":"2024-04-23T01:57:51.679265+0000","last_active":"2024-04-23T01:58:16.657609+0000","last_peered":"2024-04-23T01:58:16.657609+0000","last_clean":"2024-04-23T01:58:16.657609+0000","last_became_active":"2024-04-23T01:57:51.662859+0000","last_became_peered":"2024-04-23T01:57:51.662859+0000","last_unstale":"2024-04-23T01:58:16.657609+0000","last_undegraded":"2024-04-23T01:58:16.657609+0000","last_fullsized":"2024-04-23T01:58:16.657609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2342912,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":286,"num_write_kb":2288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"84'232","reported_seq":250,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.602770+0000","last_change":"2024-04-23T01:57:51.593118+0000","last_active":"2024-04-23T01:58:15.602770+0000","last_peered":"2024-04-23T01:58:15.602770+0000","last_clean":"2024-04-23T01:58:15.602770+0000","last_became_active":"2024-04-23T01:57:51.580553+0000","last_became_peered":"2024-04-23T01:57:51.580553+0000","last_unstale":"2024-04-23T01:58:15.602770+0000","last_undegraded":"2024-04-23T01:58:15.602770+0000","last_fullsized":"2024-04-23T01:58:15.602770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1900544,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":232,"num_write_kb":1856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"84'184","reported_seq":205,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.496287+0000","last_change":"2024-04-23T01:57:51.577069+0000","last_active":"2024-04-23T01:58:16.496287+0000","last_peered":"2024-04-23T01:58:16.496287+0000","last_clean":"2024-04-23T01:58:16.496287+0000","last_became_active":"2024-04-23T01:57:51.568227+0000","last_became_peered":"2024-04-23T01:57:51.568227+0000","last_unstale":"2024-04-23T01:58:16.496287+0000","last_undegraded":"2024-04-23T01:58:16.496287+0000","last_fullsized":"2024-04-23T01:58:16.496287+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"86'223","reported_seq":242,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.565428+0000","last_change":"2024-04-23T01:57:51.748592+0000","last_active":"2024-04-23T01:58:15.565428+0000","last_peered":"2024-04-23T01:58:15.565428+0000","last_clean":"2024-04-23T01:58:15.565428+0000","last_became_active":"2024-04-23T01:57:51.742159+0000","last_became_peered":"2024-04-23T01:57:51.742159+0000","last_unstale":"2024-04-23T01:58:15.565428+0000","last_undegraded":"2024-04-23T01:58:15.565428+0000","last_fullsized":"2024-04-23T01:58:15.565428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1835008,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":1792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"86'216","reported_seq":238,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.507137+0000","last_change":"2024-04-23T01:57:51.630095+0000","last_active":"2024-04-23T01:58:16.507137+0000","last_peered":"2024-04-23T01:58:16.507137+0000","last_clean":"2024-04-23T01:58:16.507137+0000","last_became_active":"2024-04-23T01:57:51.623783+0000","last_became_peered":"2024-04-23T01:57:51.623783+0000","last_unstale":"2024-04-23T01:58:16.507137+0000","last_undegraded":"2024-04-23T01:58:16.507137+0000","last_fullsized":"2024-04-23T01:58:16.507137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1777664,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":217,"num_write_kb":1736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"86'144","reported_seq":163,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.215058+0000","last_change":"2024-04-23T01:57:51.605374+0000","last_active":"2024-04-23T01:58:16.215058+0000","last_peered":"2024-04-23T01:58:16.215058+0000","last_clean":"2024-04-23T01:58:16.215058+0000","last_became_active":"2024-04-23T01:57:51.597053+0000","last_became_peered":"2024-04-23T01:57:51.597053+0000","last_unstale":"2024-04-23T01:58:16.215058+0000","last_undegraded":"2024-04-23T01:58:16.215058+0000","last_fullsized":"2024-04-23T01:58:16.215058+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1187840,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":145,"num_write_kb":1160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"86'287","reported_seq":306,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.026088+0000","last_change":"2024-04-23T01:57:51.641687+0000","last_active":"2024-04-23T01:58:16.026088+0000","last_peered":"2024-04-23T01:58:16.026088+0000","last_clean":"2024-04-23T01:58:16.026088+0000","last_became_active":"2024-04-23T01:57:51.623973+0000","last_became_peered":"2024-04-23T01:57:51.623973+0000","last_unstale":"2024-04-23T01:58:16.026088+0000","last_undegraded":"2024-04-23T01:58:16.026088+0000","last_fullsized":"2024-04-23T01:58:16.026088+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"84'184","reported_seq":202,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.657684+0000","last_change":"2024-04-23T01:57:51.669918+0000","last_active":"2024-04-23T01:58:15.657684+0000","last_peered":"2024-04-23T01:58:15.657684+0000","last_clean":"2024-04-23T01:58:15.657684+0000","last_became_active":"2024-04-23T01:57:51.652610+0000","last_became_peered":"2024-04-23T01:57:51.652610+0000","last_unstale":"2024-04-23T01:58:15.657684+0000","last_undegraded":"2024-04-23T01:58:15.657684+0000","last_fullsized":"2024-04-23T01:58:15.657684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":340,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.486220+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:58:16.486220+0000","last_peered":"2024-04-23T01:58:16.486220+0000","last_clean":"2024-04-23T01:58:16.486220+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:58:16.486220+0000","last_undegraded":"2024-04-23T01:58:16.486220+0000","last_fullsized":"2024-04-23T01:58:16.486220+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":206,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.674360+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:58:15.674360+0000","last_peered":"2024-04-23T01:58:15.674360+0000","last_clean":"2024-04-23T01:58:15.674360+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:58:15.674360+0000","last_undegraded":"2024-04-23T01:58:15.674360+0000","last_fullsized":"2024-04-23T01:58:15.674360+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"86'227","reported_seq":246,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.746931+0000","last_change":"2024-04-23T01:57:51.587471+0000","last_active":"2024-04-23T01:58:16.746931+0000","last_peered":"2024-04-23T01:58:16.746931+0000","last_clean":"2024-04-23T01:58:16.746931+0000","last_became_active":"2024-04-23T01:57:51.575352+0000","last_became_peered":"2024-04-23T01:57:51.575352+0000","last_unstale":"2024-04-23T01:58:16.746931+0000","last_undegraded":"2024-04-23T01:58:16.746931+0000","last_fullsized":"2024-04-23T01:58:16.746931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1867776,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":228,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":198,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.511664+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:58:15.511664+0000","last_peered":"2024-04-23T01:58:15.511664+0000","last_clean":"2024-04-23T01:58:15.511664+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:58:15.511664+0000","last_undegraded":"2024-04-23T01:58:15.511664+0000","last_fullsized":"2024-04-23T01:58:15.511664+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"87'271","reported_seq":293,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.572887+0000","last_change":"2024-04-23T01:57:51.596134+0000","last_active":"2024-04-23T01:58:16.572887+0000","last_peered":"2024-04-23T01:58:16.572887+0000","last_clean":"2024-04-23T01:58:16.572887+0000","last_became_active":"2024-04-23T01:57:51.589605+0000","last_became_peered":"2024-04-23T01:57:51.589605+0000","last_unstale":"2024-04-23T01:58:16.572887+0000","last_undegraded":"2024-04-23T01:58:16.572887+0000","last_fullsized":"2024-04-23T01:58:16.572887+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":272,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":206,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.659245+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:58:15.659245+0000","last_peered":"2024-04-23T01:58:15.659245+0000","last_clean":"2024-04-23T01:58:15.659245+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:58:15.659245+0000","last_undegraded":"2024-04-23T01:58:15.659245+0000","last_fullsized":"2024-04-23T01:58:15.659245+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"84'176","reported_seq":194,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.487735+0000","last_change":"2024-04-23T01:57:51.662237+0000","last_active":"2024-04-23T01:58:15.487735+0000","last_peered":"2024-04-23T01:58:15.487735+0000","last_clean":"2024-04-23T01:58:15.487735+0000","last_became_active":"2024-04-23T01:57:51.647583+0000","last_became_peered":"2024-04-23T01:57:51.647583+0000","last_unstale":"2024-04-23T01:58:15.487735+0000","last_undegraded":"2024-04-23T01:58:15.487735+0000","last_fullsized":"2024-04-23T01:58:15.487735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":198,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.508558+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:58:15.508558+0000","last_peered":"2024-04-23T01:58:15.508558+0000","last_clean":"2024-04-23T01:58:15.508558+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:58:15.508558+0000","last_undegraded":"2024-04-23T01:58:15.508558+0000","last_fullsized":"2024-04-23T01:58:15.508558+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"86'199","reported_seq":218,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.524373+0000","last_change":"2024-04-23T01:57:51.681957+0000","last_active":"2024-04-23T01:58:16.524373+0000","last_peered":"2024-04-23T01:58:16.524373+0000","last_clean":"2024-04-23T01:58:16.524373+0000","last_became_active":"2024-04-23T01:57:51.670788+0000","last_became_peered":"2024-04-23T01:57:51.670788+0000","last_unstale":"2024-04-23T01:58:16.524373+0000","last_undegraded":"2024-04-23T01:58:16.524373+0000","last_fullsized":"2024-04-23T01:58:16.524373+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":200,"num_write_kb":1600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":204,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.613866+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:58:15.613866+0000","last_peered":"2024-04-23T01:58:15.613866+0000","last_clean":"2024-04-23T01:58:15.613866+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:58:15.613866+0000","last_undegraded":"2024-04-23T01:58:15.613866+0000","last_fullsized":"2024-04-23T01:58:15.613866+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"86'260","reported_seq":279,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.644099+0000","last_change":"2024-04-23T01:57:51.632423+0000","last_active":"2024-04-23T01:58:16.644099+0000","last_peered":"2024-04-23T01:58:16.644099+0000","last_clean":"2024-04-23T01:58:16.644099+0000","last_became_active":"2024-04-23T01:57:51.624354+0000","last_became_peered":"2024-04-23T01:57:51.624354+0000","last_unstale":"2024-04-23T01:58:16.644099+0000","last_undegraded":"2024-04-23T01:58:16.644099+0000","last_fullsized":"2024-04-23T01:58:16.644099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2138112,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":261,"num_write_kb":2088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":207,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.499984+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:58:16.499984+0000","last_peered":"2024-04-23T01:58:16.499984+0000","last_clean":"2024-04-23T01:58:16.499984+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:58:16.499984+0000","last_undegraded":"2024-04-23T01:58:16.499984+0000","last_fullsized":"2024-04-23T01:58:16.499984+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"86'183","reported_seq":202,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:16.410714+0000","last_change":"2024-04-23T01:57:51.735746+0000","last_active":"2024-04-23T01:58:16.410714+0000","last_peered":"2024-04-23T01:58:16.410714+0000","last_clean":"2024-04-23T01:58:16.410714+0000","last_became_active":"2024-04-23T01:57:51.728785+0000","last_became_peered":"2024-04-23T01:57:51.728785+0000","last_unstale":"2024-04-23T01:58:16.410714+0000","last_undegraded":"2024-04-23T01:58:16.410714+0000","last_fullsized":"2024-04-23T01:58:16.410714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":204,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.609718+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:58:15.609718+0000","last_peered":"2024-04-23T01:58:15.609718+0000","last_clean":"2024-04-23T01:58:15.609718+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:58:15.609718+0000","last_undegraded":"2024-04-23T01:58:15.609718+0000","last_fullsized":"2024-04-23T01:58:15.609718+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"86'151","reported_seq":170,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.711529+0000","last_change":"2024-04-23T01:57:51.656207+0000","last_active":"2024-04-23T01:58:15.711529+0000","last_peered":"2024-04-23T01:58:15.711529+0000","last_clean":"2024-04-23T01:58:15.711529+0000","last_became_active":"2024-04-23T01:57:51.638689+0000","last_became_peered":"2024-04-23T01:57:51.638689+0000","last_unstale":"2024-04-23T01:58:15.711529+0000","last_undegraded":"2024-04-23T01:58:15.711529+0000","last_fullsized":"2024-04-23T01:58:15.711529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":206,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T01:58:15.661790+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:58:15.661790+0000","last_peered":"2024-04-23T01:58:15.661790+0000","last_clean":"2024-04-23T01:58:15.661790+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:58:15.661790+0000","last_undegraded":"2024-04-23T01:58:15.661790+0000","last_fullsized":"2024-04-23T01:58:15.661790+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":28516352,"num_objects":438,"num_object_clones":0,"num_object_copies":1314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3481,"num_write_kb":27848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":42786816,"data_stored":85573632,"data_compressed":4415328,"data_compressed_allocated":42786816,"data_compressed_original":85573632,"omap_allocated":0,"internal_metadata":0},"log_size":3469,"ondisk_log_size":3469,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706453,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":129952,"kb_used_data":8728,"kb_used_omap":0,"kb_used_meta":121216,"kb_avail":94241888,"statfs":{"total":96636764160,"available":96503693312,"internally_reserved":0,"allocated":8937472,"data_stored":17126972,"data_compressed":869012,"data_compressed_allocated":8171520,"data_compressed_original":16834560,"omap_allocated":0,"internal_metadata":124125184},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706453,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":115856,"kb_used_data":12680,"kb_used_omap":0,"kb_used_meta":103168,"kb_avail":94255984,"statfs":{"total":96636764160,"available":96518127616,"internally_reserved":0,"allocated":12984320,"data_stored":25233807,"data_compressed":1287388,"data_compressed_allocated":12226560,"data_compressed_original":24944640,"omap_allocated":0,"internal_metadata":105644032},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706452,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":131860,"kb_used_data":10636,"kb_used_omap":0,"kb_used_meta":121216,"kb_avail":94239980,"statfs":{"total":96636764160,"available":96501739520,"internally_reserved":0,"allocated":10891264,"data_stored":20580207,"data_compressed":1047731,"data_compressed_allocated":10153984,"data_compressed_original":20307968,"omap_allocated":0,"internal_metadata":124125184},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706452,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":141400,"kb_used_data":11856,"kb_used_omap":0,"kb_used_meta":129536,"kb_avail":94230440,"statfs":{"total":96636764160,"available":96491970560,"internally_reserved":0,"allocated":12140544,"data_stored":23546255,"data_compressed":1200382,"data_compressed_allocated":11382784,"data_compressed_original":23257088,"omap_allocated":0,"internal_metadata":132644864},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":11776000,"data_stored":23552000,"data_compressed":1215267,"data_compressed_allocated":11776000,"data_compressed_original":23552000,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10575872,"data_stored":21151744,"data_compressed":1091300,"data_compressed_allocated":10575872,"data_compressed_original":21151744,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12292096,"data_stored":24584192,"data_compressed":1268433,"data_compressed_allocated":12292096,"data_compressed_original":24584192,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8142848,"data_stored":16285696,"data_compressed":840328,"data_compressed_allocated":8142848,"data_compressed_original":16285696,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:58:19.483 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 27 16 3872 3856 1.11564 1.05469 0.147907 0.111703 2024-04-23T01:58:19.771 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:58:19.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:19.883+0000 7fe2941ff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:19.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:19.891+0000 7fe292f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:20.483 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 28 16 4010 3994 1.1143 1.07812 0.0639836 0.111925 2024-04-23T01:58:21.483 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 29 16 4164 4148 1.11736 1.20312 0.140748 0.111563 2024-04-23T01:58:22.062 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:58:22.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:22.212+0000 7f01104ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:22.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:22.218+0000 7f01104ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:22.483 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 30 16 4307 4291 1.11735 1.11719 0.100896 0.111564 2024-04-23T01:58:23.483 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 31 16 4447 4431 1.11658 1.09375 0.0957007 0.111713 2024-04-23T01:58:23.768 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:58:23.768 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:58:23.769 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:58:23.769 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:58:23.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:23.888+0000 7fc8a786d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:23.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:23.902+0000 7fc8a786d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:23.979 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:58:23.979 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:58:23.979 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:58:23.979 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:58:23.979 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:58:23.979 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:58:23.980 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:58:23.980 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:58:23.980 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:58:23.980 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:58:23.980 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:58:23.980 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:58:23.982 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:58:23.983 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:58:23.983 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:58:24.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:24.100+0000 7f57b7554640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:24.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:24.111+0000 7f57b7554640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:24.160 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:58:24.160 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:58:24.160 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:58:24.160 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:58:24.160 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:58:24.160 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:58:24.160 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:58:24.161 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:58:24.161 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:58:24.161 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:58:24.161 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:58:24.161 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:58:24.164 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:58:24.164 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:58:24.164 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:58:24.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:24.288+0000 7fd9f335d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:24.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:24.302+0000 7fd9f335d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:24.361 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:58:24.367 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:58:24.367 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:58:24.367 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:58:24.367 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:58:24.367 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:58:24.368 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:58:24.368 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:58:24.368 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:58:24.368 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:58:24.368 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:58:24.368 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:58:24.368 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:58:24.371 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:58:24.372 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:58:24.372 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:58:24.483 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 32 16 4576 4560 1.11318 1.00781 0.0711151 0.11203 2024-04-23T01:58:24.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:24.513+0000 7f3c6e42d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:24.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:24.520+0000 7f3c6e42d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:24.542+0000 7f058615d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:24.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:24.555+0000 7f058615d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:24.588 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:58:24.588 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:58:24.588 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:58:24.588 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:58:24.588 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:58:24.588 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:58:24.588 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:58:24.589 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:58:24.589 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:58:24.589 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:58:24.589 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:58:24.589 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:58:24.592 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:58:24.592 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:58:25.483 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 33 16 4716 4700 1.11259 1.09375 0.075966 0.112142 2024-04-23T01:58:26.484 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 34 16 4859 4843 1.11272 1.11719 0.0973025 0.112115 2024-04-23T01:58:26.731 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:58:26.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:26.851+0000 7f21b799d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:26.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:26.859+0000 7f21b799d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:27.484 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 35 16 5018 5002 1.11642 1.24219 0.0799494 0.111827 2024-04-23T01:58:28.484 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 36 16 5166 5150 1.11752 1.15625 0.112161 0.11165 2024-04-23T01:58:29.067 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:58:29.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:29.214+0000 7f4a7efff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:29.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:29.221+0000 7f4a7efff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:29.484 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 37 16 5294 5278 1.11434 1 0.13996 0.111944 2024-04-23T01:58:29.594 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:58:29.594 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:58:29.594 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:58:29.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:29.708+0000 7fe6b90ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:29.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:29.717+0000 7fe6b90ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:29.940 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:58:29.941 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:58:29.953 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":483,"stamp":"2024-04-23T01:58:29.705517+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":40616480,"num_objects":615,"num_object_clones":0,"num_object_copies":1845,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":5023,"num_write_kb":40982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4967,"ondisk_log_size":4967,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":535968,"kb_used_data":60800,"kb_used_omap":0,"kb_used_meta":475136,"kb_avail":376951392,"statfs":{"total":386547056640,"available":385998225408,"internally_reserved":0,"allocated":62259200,"data_stored":121006468,"data_compressed":6185766,"data_compressed_allocated":59183104,"data_compressed_original":119840768,"omap_allocated":0,"internal_metadata":486539264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11509760,"num_objects":175,"num_object_clones":0,"num_object_copies":525,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1405,"num_write_kb":11240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002479"},"pg_stats":[{"pgid":"10.7","version":"88'320","reported_seq":345,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:26.594649+0000","last_change":"2024-04-23T01:57:51.583793+0000","last_active":"2024-04-23T01:58:26.594649+0000","last_peered":"2024-04-23T01:58:26.594649+0000","last_clean":"2024-04-23T01:58:26.594649+0000","last_became_active":"2024-04-23T01:57:51.574869+0000","last_became_peered":"2024-04-23T01:57:51.574869+0000","last_unstale":"2024-04-23T01:58:26.594649+0000","last_undegraded":"2024-04-23T01:58:26.594649+0000","last_fullsized":"2024-04-23T01:58:26.594649+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2629632,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":321,"num_write_kb":2568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"88'343","reported_seq":366,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:25.611175+0000","last_change":"2024-04-23T01:57:51.679265+0000","last_active":"2024-04-23T01:58:25.611175+0000","last_peered":"2024-04-23T01:58:25.611175+0000","last_clean":"2024-04-23T01:58:25.611175+0000","last_became_active":"2024-04-23T01:57:51.662859+0000","last_became_peered":"2024-04-23T01:57:51.662859+0000","last_unstale":"2024-04-23T01:58:25.611175+0000","last_undegraded":"2024-04-23T01:58:25.611175+0000","last_fullsized":"2024-04-23T01:58:25.611175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":344,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"88'279","reported_seq":302,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:25.975786+0000","last_change":"2024-04-23T01:57:51.593118+0000","last_active":"2024-04-23T01:58:25.975786+0000","last_peered":"2024-04-23T01:58:25.975786+0000","last_clean":"2024-04-23T01:58:25.975786+0000","last_became_active":"2024-04-23T01:57:51.580553+0000","last_became_peered":"2024-04-23T01:57:51.580553+0000","last_unstale":"2024-04-23T01:58:25.975786+0000","last_undegraded":"2024-04-23T01:58:25.975786+0000","last_fullsized":"2024-04-23T01:58:25.975786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2293760,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":280,"num_write_kb":2240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"88'263","reported_seq":288,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:26.155074+0000","last_change":"2024-04-23T01:57:51.577069+0000","last_active":"2024-04-23T01:58:26.155074+0000","last_peered":"2024-04-23T01:58:26.155074+0000","last_clean":"2024-04-23T01:58:26.155074+0000","last_became_active":"2024-04-23T01:57:51.568227+0000","last_became_peered":"2024-04-23T01:57:51.568227+0000","last_unstale":"2024-04-23T01:58:26.155074+0000","last_undegraded":"2024-04-23T01:58:26.155074+0000","last_fullsized":"2024-04-23T01:58:26.155074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162688,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":264,"num_write_kb":2112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"88'382","reported_seq":407,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:26.654362+0000","last_change":"2024-04-23T01:57:51.748592+0000","last_active":"2024-04-23T01:58:26.654362+0000","last_peered":"2024-04-23T01:58:26.654362+0000","last_clean":"2024-04-23T01:58:26.654362+0000","last_became_active":"2024-04-23T01:57:51.742159+0000","last_became_peered":"2024-04-23T01:57:51.742159+0000","last_unstale":"2024-04-23T01:58:26.654362+0000","last_undegraded":"2024-04-23T01:58:26.654362+0000","last_fullsized":"2024-04-23T01:58:26.654362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3137536,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":383,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"88'342","reported_seq":367,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:26.715213+0000","last_change":"2024-04-23T01:57:51.630095+0000","last_active":"2024-04-23T01:58:26.715213+0000","last_peered":"2024-04-23T01:58:26.715213+0000","last_clean":"2024-04-23T01:58:26.715213+0000","last_became_active":"2024-04-23T01:57:51.623783+0000","last_became_peered":"2024-04-23T01:57:51.623783+0000","last_unstale":"2024-04-23T01:58:26.715213+0000","last_undegraded":"2024-04-23T01:58:26.715213+0000","last_fullsized":"2024-04-23T01:58:26.715213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2809856,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":343,"num_write_kb":2744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"88'247","reported_seq":272,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:25.083646+0000","last_change":"2024-04-23T01:57:51.605374+0000","last_active":"2024-04-23T01:58:25.083646+0000","last_peered":"2024-04-23T01:58:25.083646+0000","last_clean":"2024-04-23T01:58:25.083646+0000","last_became_active":"2024-04-23T01:57:51.597053+0000","last_became_peered":"2024-04-23T01:57:51.597053+0000","last_unstale":"2024-04-23T01:58:25.083646+0000","last_undegraded":"2024-04-23T01:58:25.083646+0000","last_fullsized":"2024-04-23T01:58:25.083646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"88'415","reported_seq":438,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:26.611815+0000","last_change":"2024-04-23T01:57:51.641687+0000","last_active":"2024-04-23T01:58:26.611815+0000","last_peered":"2024-04-23T01:58:26.611815+0000","last_clean":"2024-04-23T01:58:26.611815+0000","last_became_active":"2024-04-23T01:57:51.623973+0000","last_became_peered":"2024-04-23T01:57:51.623973+0000","last_unstale":"2024-04-23T01:58:26.611815+0000","last_undegraded":"2024-04-23T01:58:26.611815+0000","last_fullsized":"2024-04-23T01:58:26.611815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3407872,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":3328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"88'251","reported_seq":274,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:26.663745+0000","last_change":"2024-04-23T01:57:51.669918+0000","last_active":"2024-04-23T01:58:26.663745+0000","last_peered":"2024-04-23T01:58:26.663745+0000","last_clean":"2024-04-23T01:58:26.663745+0000","last_became_active":"2024-04-23T01:57:51.652610+0000","last_became_peered":"2024-04-23T01:57:51.652610+0000","last_unstale":"2024-04-23T01:58:26.663745+0000","last_undegraded":"2024-04-23T01:58:26.663745+0000","last_fullsized":"2024-04-23T01:58:26.663745+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2064384,"num_objects":32,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":252,"num_write_kb":2016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":343,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:17.698235+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:58:17.698235+0000","last_peered":"2024-04-23T01:58:17.698235+0000","last_clean":"2024-04-23T01:58:17.698235+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:58:17.698235+0000","last_undegraded":"2024-04-23T01:58:17.698235+0000","last_fullsized":"2024-04-23T01:58:17.698235+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":210,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:17.792601+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:58:17.792601+0000","last_peered":"2024-04-23T01:58:17.792601+0000","last_clean":"2024-04-23T01:58:17.792601+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:58:17.792601+0000","last_undegraded":"2024-04-23T01:58:17.792601+0000","last_fullsized":"2024-04-23T01:58:17.792601+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"88'290","reported_seq":313,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:26.747171+0000","last_change":"2024-04-23T01:57:51.587471+0000","last_active":"2024-04-23T01:58:26.747171+0000","last_peered":"2024-04-23T01:58:26.747171+0000","last_clean":"2024-04-23T01:58:26.747171+0000","last_became_active":"2024-04-23T01:57:51.575352+0000","last_became_peered":"2024-04-23T01:57:51.575352+0000","last_unstale":"2024-04-23T01:58:26.747171+0000","last_undegraded":"2024-04-23T01:58:26.747171+0000","last_fullsized":"2024-04-23T01:58:26.747171+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2383872,"num_objects":37,"num_object_clones":0,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":291,"num_write_kb":2328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":204,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:17.809011+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:58:17.809011+0000","last_peered":"2024-04-23T01:58:17.809011+0000","last_clean":"2024-04-23T01:58:17.809011+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:58:17.809011+0000","last_undegraded":"2024-04-23T01:58:17.809011+0000","last_fullsized":"2024-04-23T01:58:17.809011+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"88'367","reported_seq":392,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:26.434299+0000","last_change":"2024-04-23T01:57:51.596134+0000","last_active":"2024-04-23T01:58:26.434299+0000","last_peered":"2024-04-23T01:58:26.434299+0000","last_clean":"2024-04-23T01:58:26.434299+0000","last_became_active":"2024-04-23T01:57:51.589605+0000","last_became_peered":"2024-04-23T01:57:51.589605+0000","last_unstale":"2024-04-23T01:58:26.434299+0000","last_undegraded":"2024-04-23T01:58:26.434299+0000","last_fullsized":"2024-04-23T01:58:26.434299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3014656,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":368,"num_write_kb":2944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":210,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:17.788210+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:58:17.788210+0000","last_peered":"2024-04-23T01:58:17.788210+0000","last_clean":"2024-04-23T01:58:17.788210+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:58:17.788210+0000","last_undegraded":"2024-04-23T01:58:17.788210+0000","last_fullsized":"2024-04-23T01:58:17.788210+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"88'271","reported_seq":296,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:26.074261+0000","last_change":"2024-04-23T01:57:51.662237+0000","last_active":"2024-04-23T01:58:26.074261+0000","last_peered":"2024-04-23T01:58:26.074261+0000","last_clean":"2024-04-23T01:58:26.074261+0000","last_became_active":"2024-04-23T01:57:51.647583+0000","last_became_peered":"2024-04-23T01:57:51.647583+0000","last_unstale":"2024-04-23T01:58:26.074261+0000","last_undegraded":"2024-04-23T01:58:26.074261+0000","last_fullsized":"2024-04-23T01:58:26.074261+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":272,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":204,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:17.809489+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:58:17.809489+0000","last_peered":"2024-04-23T01:58:17.809489+0000","last_clean":"2024-04-23T01:58:17.809489+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:58:17.809489+0000","last_undegraded":"2024-04-23T01:58:17.809489+0000","last_fullsized":"2024-04-23T01:58:17.809489+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"88'303","reported_seq":328,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:26.360144+0000","last_change":"2024-04-23T01:57:51.681957+0000","last_active":"2024-04-23T01:58:26.360144+0000","last_peered":"2024-04-23T01:58:26.360144+0000","last_clean":"2024-04-23T01:58:26.360144+0000","last_became_active":"2024-04-23T01:57:51.670788+0000","last_became_peered":"2024-04-23T01:57:51.670788+0000","last_unstale":"2024-04-23T01:58:26.360144+0000","last_undegraded":"2024-04-23T01:58:26.360144+0000","last_fullsized":"2024-04-23T01:58:26.360144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490368,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":208,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:17.516673+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:58:17.516673+0000","last_peered":"2024-04-23T01:58:17.516673+0000","last_clean":"2024-04-23T01:58:17.516673+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:58:17.516673+0000","last_undegraded":"2024-04-23T01:58:17.516673+0000","last_fullsized":"2024-04-23T01:58:17.516673+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"88'311","reported_seq":336,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:25.964108+0000","last_change":"2024-04-23T01:57:51.632423+0000","last_active":"2024-04-23T01:58:25.964108+0000","last_peered":"2024-04-23T01:58:25.964108+0000","last_clean":"2024-04-23T01:58:25.964108+0000","last_became_active":"2024-04-23T01:57:51.624354+0000","last_became_peered":"2024-04-23T01:57:51.624354+0000","last_unstale":"2024-04-23T01:58:25.964108+0000","last_undegraded":"2024-04-23T01:58:25.964108+0000","last_fullsized":"2024-04-23T01:58:25.964108+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2555904,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":2496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":210,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:17.712052+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:58:17.712052+0000","last_peered":"2024-04-23T01:58:17.712052+0000","last_clean":"2024-04-23T01:58:17.712052+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:58:17.712052+0000","last_undegraded":"2024-04-23T01:58:17.712052+0000","last_fullsized":"2024-04-23T01:58:17.712052+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"88'271","reported_seq":296,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:26.234459+0000","last_change":"2024-04-23T01:57:51.735746+0000","last_active":"2024-04-23T01:58:26.234459+0000","last_peered":"2024-04-23T01:58:26.234459+0000","last_clean":"2024-04-23T01:58:26.234459+0000","last_became_active":"2024-04-23T01:57:51.728785+0000","last_became_peered":"2024-04-23T01:57:51.728785+0000","last_unstale":"2024-04-23T01:58:26.234459+0000","last_undegraded":"2024-04-23T01:58:26.234459+0000","last_fullsized":"2024-04-23T01:58:26.234459+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":272,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":208,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:17.527781+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:58:17.527781+0000","last_peered":"2024-04-23T01:58:17.527781+0000","last_clean":"2024-04-23T01:58:17.527781+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:58:17.527781+0000","last_undegraded":"2024-04-23T01:58:17.527781+0000","last_fullsized":"2024-04-23T01:58:17.527781+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"88'215","reported_seq":238,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:26.081760+0000","last_change":"2024-04-23T01:57:51.656207+0000","last_active":"2024-04-23T01:58:26.081760+0000","last_peered":"2024-04-23T01:58:26.081760+0000","last_clean":"2024-04-23T01:58:26.081760+0000","last_became_active":"2024-04-23T01:57:51.638689+0000","last_became_peered":"2024-04-23T01:57:51.638689+0000","last_unstale":"2024-04-23T01:58:26.081760+0000","last_undegraded":"2024-04-23T01:58:26.081760+0000","last_fullsized":"2024-04-23T01:58:26.081760+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1769472,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":216,"num_write_kb":1728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":210,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:17.794113+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:58:17.794113+0000","last_peered":"2024-04-23T01:58:17.794113+0000","last_clean":"2024-04-23T01:58:17.794113+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:58:17.794113+0000","last_undegraded":"2024-04-23T01:58:17.794113+0000","last_fullsized":"2024-04-23T01:58:17.794113+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":40026112,"num_objects":613,"num_object_clones":0,"num_object_copies":1839,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4886,"num_write_kb":39088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":60059648,"data_stored":120119296,"data_compressed":6199119,"data_compressed_allocated":60059648,"data_compressed_original":120119296,"omap_allocated":0,"internal_metadata":0},"log_size":4870,"ondisk_log_size":4870,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706463,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":133240,"kb_used_data":12016,"kb_used_omap":0,"kb_used_meta":121216,"kb_avail":94238600,"statfs":{"total":96636764160,"available":96500326400,"internally_reserved":0,"allocated":12304384,"data_stored":23847657,"data_compressed":1215872,"data_compressed_allocated":11530240,"data_compressed_original":23552000,"omap_allocated":0,"internal_metadata":124125184},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706463,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":124724,"kb_used_data":17388,"kb_used_omap":0,"kb_used_meta":107328,"kb_avail":94247116,"statfs":{"total":96636764160,"available":96509046784,"internally_reserved":0,"allocated":17805312,"data_stored":34849513,"data_compressed":1783567,"data_compressed_allocated":17031168,"data_compressed_original":34553856,"omap_allocated":0,"internal_metadata":109903872},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":140076,"kb_used_data":14692,"kb_used_omap":0,"kb_used_meta":125376,"kb_avail":94231764,"statfs":{"total":96636764160,"available":96493326336,"internally_reserved":0,"allocated":15044608,"data_stored":28860617,"data_compressed":1474961,"data_compressed_allocated":14290944,"data_compressed_original":28581888,"omap_allocated":0,"internal_metadata":128385024},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":137928,"kb_used_data":16704,"kb_used_omap":0,"kb_used_meta":121216,"kb_avail":94233912,"statfs":{"total":96636764160,"available":96495525888,"internally_reserved":0,"allocated":17104896,"data_stored":33448681,"data_compressed":1711366,"data_compressed_allocated":16330752,"data_compressed_original":33153024,"omap_allocated":0,"internal_metadata":124125184},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16723968,"data_stored":33447936,"data_compressed":1726251,"data_compressed_allocated":16723968,"data_compressed_original":33447936,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14684160,"data_stored":29368320,"data_compressed":1515569,"data_compressed_allocated":14684160,"data_compressed_original":29368320,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":17141760,"data_stored":34283520,"data_compressed":1769265,"data_compressed_allocated":17141760,"data_compressed_original":34283520,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11509760,"data_stored":23019520,"data_compressed":1188034,"data_compressed_allocated":11509760,"data_compressed_original":23019520,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:58:30.484 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 38 16 5430 5414 1.11298 1.0625 0.123675 0.112119 2024-04-23T01:58:31.367 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:58:31.484 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 39 16 5564 5548 1.11128 1.04688 0.130336 0.112328 2024-04-23T01:58:31.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:31.487+0000 7f19e5c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:31.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:31.494+0000 7f19e5c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:32.484 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:58:32.483363+0000 min lat: 0.0138177 max lat: 0.388479 avg lat: 0.112264 2024-04-23T01:58:32.484 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:58:32.484 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 40 16 5703 5687 1.11064 1.08594 0.0288499 0.112264 2024-04-23T01:58:33.484 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 41 16 5838 5822 1.10928 1.05469 0.103755 0.112476 2024-04-23T01:58:33.759 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:58:33.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:33.875+0000 7f6ae8d5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:33.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:33.882+0000 7f6ae8d5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:34.484 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 42 16 5947 5931 1.10314 0.851562 0.14116 0.113152 2024-04-23T01:58:34.954 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:58:34.955 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-23T01:58:35.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:35.060+0000 7fa38f66d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:35.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:35.071+0000 7fa38f66d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:35.484 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 43 16 6069 6053 1.09965 0.953125 0.134546 0.113455 2024-04-23T01:58:36.098 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:58:36.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:36.226+0000 7fd96aaad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:36.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:36.232+0000 7fd96aaad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:36.275 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.0 to 1 (10000) 2024-04-23T01:58:36.291 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-23T01:58:36.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:36.414+0000 7fb5f7d5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:36.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:36.428+0000 7fb5f7d5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:36.484 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 44 16 6218 6202 1.10111 1.16406 0.266197 0.113316 2024-04-23T01:58:37.276 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.1 to 1 (10000) 2024-04-23T01:58:37.295 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-23T01:58:37.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:37.461+0000 7ff3b71ff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:37.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:37.473+0000 7ff3b5f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:37.484 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 45 16 6364 6348 1.10199 1.14062 0.0951031 0.113294 2024-04-23T01:58:38.281 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.2 to 1 (10000) 2024-04-23T01:58:38.297 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-23T01:58:38.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:38.430+0000 7f1157e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:38.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:38.444+0000 7f1157e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:38.466 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:58:38.484 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 46 16 6493 6477 1.09994 1.00781 0.0583716 0.113397 2024-04-23T01:58:38.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:38.619+0000 7f6585d54640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:38.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:38.625+0000 7f6585d54640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:39.250 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.3 to 1 (10000) 2024-04-23T01:58:39.265 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-23T01:58:39.266 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:58:39.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:39.377+0000 7fcc0315d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:39.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:39.387+0000 7fcc0315d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:39.485 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 47 16 6622 6606 1.09797 1.00781 0.101177 0.113636 2024-04-23T01:58:39.599 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:58:39.600 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:58:39.622 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":491,"stamp":"2024-04-23T01:58:39.250075+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":51773984,"num_objects":786,"num_object_clones":0,"num_object_copies":2358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":6385,"num_write_kb":51878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6330,"ondisk_log_size":6330,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":568952,"kb_used_data":77144,"kb_used_omap":0,"kb_used_meta":491776,"kb_avail":376918408,"statfs":{"total":386547056640,"available":385964449792,"internally_reserved":0,"allocated":78995456,"data_stored":154465841,"data_compressed":7913298,"data_compressed_allocated":75911168,"data_compressed_original":153296896,"omap_allocated":0,"internal_metadata":503578624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5480448,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":669,"num_write_kb":5352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.543803"},"pg_stats":[{"pgid":"10.7","version":"88'415","reported_seq":440,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:36.601627+0000","last_change":"2024-04-23T01:57:51.583793+0000","last_active":"2024-04-23T01:58:36.601627+0000","last_peered":"2024-04-23T01:58:36.601627+0000","last_clean":"2024-04-23T01:58:36.601627+0000","last_became_active":"2024-04-23T01:57:51.574869+0000","last_became_peered":"2024-04-23T01:57:51.574869+0000","last_unstale":"2024-04-23T01:58:36.601627+0000","last_undegraded":"2024-04-23T01:58:36.601627+0000","last_fullsized":"2024-04-23T01:58:36.601627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3407872,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":3328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"88'407","reported_seq":430,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:36.150983+0000","last_change":"2024-04-23T01:57:51.679265+0000","last_active":"2024-04-23T01:58:36.150983+0000","last_peered":"2024-04-23T01:58:36.150983+0000","last_clean":"2024-04-23T01:58:36.150983+0000","last_became_active":"2024-04-23T01:57:51.662859+0000","last_became_peered":"2024-04-23T01:57:51.662859+0000","last_unstale":"2024-04-23T01:58:36.150983+0000","last_undegraded":"2024-04-23T01:58:36.150983+0000","last_fullsized":"2024-04-23T01:58:36.150983+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342336,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":408,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"88'351","reported_seq":374,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:36.639614+0000","last_change":"2024-04-23T01:57:51.593118+0000","last_active":"2024-04-23T01:58:36.639614+0000","last_peered":"2024-04-23T01:58:36.639614+0000","last_clean":"2024-04-23T01:58:36.639614+0000","last_became_active":"2024-04-23T01:57:51.580553+0000","last_became_peered":"2024-04-23T01:57:51.580553+0000","last_unstale":"2024-04-23T01:58:36.639614+0000","last_undegraded":"2024-04-23T01:58:36.639614+0000","last_fullsized":"2024-04-23T01:58:36.639614+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883584,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":352,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"88'328","reported_seq":355,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T01:58:36.306137+0000","last_change":"2024-04-23T01:57:51.577069+0000","last_active":"2024-04-23T01:58:36.306137+0000","last_peered":"2024-04-23T01:58:36.306137+0000","last_clean":"2024-04-23T01:58:36.306137+0000","last_became_active":"2024-04-23T01:57:51.568227+0000","last_became_peered":"2024-04-23T01:57:51.568227+0000","last_unstale":"2024-04-23T01:58:36.306137+0000","last_undegraded":"2024-04-23T01:58:36.306137+0000","last_fullsized":"2024-04-23T01:58:36.306137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2686976,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":328,"num_write_kb":2624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"88'468","reported_seq":493,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:36.655084+0000","last_change":"2024-04-23T01:57:51.748592+0000","last_active":"2024-04-23T01:58:36.655084+0000","last_peered":"2024-04-23T01:58:36.655084+0000","last_clean":"2024-04-23T01:58:36.655084+0000","last_became_active":"2024-04-23T01:57:51.742159+0000","last_became_peered":"2024-04-23T01:57:51.742159+0000","last_unstale":"2024-04-23T01:58:36.655084+0000","last_undegraded":"2024-04-23T01:58:36.655084+0000","last_fullsized":"2024-04-23T01:58:36.655084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842048,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":469,"num_write_kb":3752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"89'407","reported_seq":435,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T01:58:36.400059+0000","last_change":"2024-04-23T01:57:51.630095+0000","last_active":"2024-04-23T01:58:36.400059+0000","last_peered":"2024-04-23T01:58:36.400059+0000","last_clean":"2024-04-23T01:58:36.400059+0000","last_became_active":"2024-04-23T01:57:51.623783+0000","last_became_peered":"2024-04-23T01:57:51.623783+0000","last_unstale":"2024-04-23T01:58:36.400059+0000","last_undegraded":"2024-04-23T01:58:36.400059+0000","last_fullsized":"2024-04-23T01:58:36.400059+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342336,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":408,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"88'327","reported_seq":352,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:36.382514+0000","last_change":"2024-04-23T01:57:51.605374+0000","last_active":"2024-04-23T01:58:36.382514+0000","last_peered":"2024-04-23T01:58:36.382514+0000","last_clean":"2024-04-23T01:58:36.382514+0000","last_became_active":"2024-04-23T01:57:51.597053+0000","last_became_peered":"2024-04-23T01:57:51.597053+0000","last_unstale":"2024-04-23T01:58:36.382514+0000","last_undegraded":"2024-04-23T01:58:36.382514+0000","last_fullsized":"2024-04-23T01:58:36.382514+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2686976,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":328,"num_write_kb":2624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"88'530","reported_seq":553,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:36.683830+0000","last_change":"2024-04-23T01:57:51.641687+0000","last_active":"2024-04-23T01:58:36.683830+0000","last_peered":"2024-04-23T01:58:36.683830+0000","last_clean":"2024-04-23T01:58:36.683830+0000","last_became_active":"2024-04-23T01:57:51.623973+0000","last_became_peered":"2024-04-23T01:57:51.623973+0000","last_unstale":"2024-04-23T01:58:36.683830+0000","last_undegraded":"2024-04-23T01:58:36.683830+0000","last_fullsized":"2024-04-23T01:58:36.683830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4349952,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":531,"num_write_kb":4248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"88'327","reported_seq":350,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:35.623029+0000","last_change":"2024-04-23T01:57:51.669918+0000","last_active":"2024-04-23T01:58:35.623029+0000","last_peered":"2024-04-23T01:58:35.623029+0000","last_clean":"2024-04-23T01:58:35.623029+0000","last_became_active":"2024-04-23T01:57:51.652610+0000","last_became_peered":"2024-04-23T01:57:51.652610+0000","last_unstale":"2024-04-23T01:58:35.623029+0000","last_undegraded":"2024-04-23T01:58:35.623029+0000","last_fullsized":"2024-04-23T01:58:35.623029+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2686976,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":328,"num_write_kb":2624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":346,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T01:58:36.304591+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:58:36.304591+0000","last_peered":"2024-04-23T01:58:36.304591+0000","last_clean":"2024-04-23T01:58:36.304591+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:58:36.304591+0000","last_undegraded":"2024-04-23T01:58:36.304591+0000","last_fullsized":"2024-04-23T01:58:36.304591+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":210,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:17.792601+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:58:17.792601+0000","last_peered":"2024-04-23T01:58:17.792601+0000","last_clean":"2024-04-23T01:58:17.792601+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:58:17.792601+0000","last_undegraded":"2024-04-23T01:58:17.792601+0000","last_fullsized":"2024-04-23T01:58:17.792601+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"88'416","reported_seq":439,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:36.000916+0000","last_change":"2024-04-23T01:57:51.587471+0000","last_active":"2024-04-23T01:58:36.000916+0000","last_peered":"2024-04-23T01:58:36.000916+0000","last_clean":"2024-04-23T01:58:36.000916+0000","last_became_active":"2024-04-23T01:57:51.575352+0000","last_became_peered":"2024-04-23T01:57:51.575352+0000","last_unstale":"2024-04-23T01:58:36.000916+0000","last_undegraded":"2024-04-23T01:58:36.000916+0000","last_fullsized":"2024-04-23T01:58:36.000916+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3416064,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":417,"num_write_kb":3336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":204,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:17.809011+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:58:17.809011+0000","last_peered":"2024-04-23T01:58:17.809011+0000","last_clean":"2024-04-23T01:58:17.809011+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:58:17.809011+0000","last_undegraded":"2024-04-23T01:58:17.809011+0000","last_fullsized":"2024-04-23T01:58:17.809011+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"89'474","reported_seq":502,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T01:58:36.728139+0000","last_change":"2024-04-23T01:57:51.596134+0000","last_active":"2024-04-23T01:58:36.728139+0000","last_peered":"2024-04-23T01:58:36.728139+0000","last_clean":"2024-04-23T01:58:36.728139+0000","last_became_active":"2024-04-23T01:57:51.589605+0000","last_became_peered":"2024-04-23T01:57:51.589605+0000","last_unstale":"2024-04-23T01:58:36.728139+0000","last_undegraded":"2024-04-23T01:58:36.728139+0000","last_fullsized":"2024-04-23T01:58:36.728139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3891200,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":475,"num_write_kb":3800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":210,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:17.788210+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:58:17.788210+0000","last_peered":"2024-04-23T01:58:17.788210+0000","last_clean":"2024-04-23T01:58:17.788210+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:58:17.788210+0000","last_undegraded":"2024-04-23T01:58:17.788210+0000","last_fullsized":"2024-04-23T01:58:17.788210+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"88'351","reported_seq":376,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:36.121564+0000","last_change":"2024-04-23T01:57:51.662237+0000","last_active":"2024-04-23T01:58:36.121564+0000","last_peered":"2024-04-23T01:58:36.121564+0000","last_clean":"2024-04-23T01:58:36.121564+0000","last_became_active":"2024-04-23T01:57:51.647583+0000","last_became_peered":"2024-04-23T01:57:51.647583+0000","last_unstale":"2024-04-23T01:58:36.121564+0000","last_undegraded":"2024-04-23T01:58:36.121564+0000","last_fullsized":"2024-04-23T01:58:36.121564+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883584,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":352,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":204,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:17.809489+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:58:17.809489+0000","last_peered":"2024-04-23T01:58:17.809489+0000","last_clean":"2024-04-23T01:58:17.809489+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:58:17.809489+0000","last_undegraded":"2024-04-23T01:58:17.809489+0000","last_fullsized":"2024-04-23T01:58:17.809489+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"88'367","reported_seq":392,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:35.391115+0000","last_change":"2024-04-23T01:57:51.681957+0000","last_active":"2024-04-23T01:58:35.391115+0000","last_peered":"2024-04-23T01:58:35.391115+0000","last_clean":"2024-04-23T01:58:35.391115+0000","last_became_active":"2024-04-23T01:57:51.670788+0000","last_became_peered":"2024-04-23T01:57:51.670788+0000","last_unstale":"2024-04-23T01:58:35.391115+0000","last_undegraded":"2024-04-23T01:58:35.391115+0000","last_fullsized":"2024-04-23T01:58:35.391115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3014656,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":368,"num_write_kb":2944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":208,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:17.516673+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:58:17.516673+0000","last_peered":"2024-04-23T01:58:17.516673+0000","last_clean":"2024-04-23T01:58:17.516673+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:58:17.516673+0000","last_undegraded":"2024-04-23T01:58:17.516673+0000","last_fullsized":"2024-04-23T01:58:17.516673+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"88'395","reported_seq":420,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:36.641061+0000","last_change":"2024-04-23T01:57:51.632423+0000","last_active":"2024-04-23T01:58:36.641061+0000","last_peered":"2024-04-23T01:58:36.641061+0000","last_clean":"2024-04-23T01:58:36.641061+0000","last_became_active":"2024-04-23T01:57:51.624354+0000","last_became_peered":"2024-04-23T01:57:51.624354+0000","last_unstale":"2024-04-23T01:58:36.641061+0000","last_undegraded":"2024-04-23T01:58:36.641061+0000","last_fullsized":"2024-04-23T01:58:36.641061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3244032,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":396,"num_write_kb":3168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":213,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T01:58:36.310414+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:58:36.310414+0000","last_peered":"2024-04-23T01:58:36.310414+0000","last_clean":"2024-04-23T01:58:36.310414+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:58:36.310414+0000","last_undegraded":"2024-04-23T01:58:36.310414+0000","last_fullsized":"2024-04-23T01:58:36.310414+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"88'375","reported_seq":400,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:35.678044+0000","last_change":"2024-04-23T01:57:51.735746+0000","last_active":"2024-04-23T01:58:35.678044+0000","last_peered":"2024-04-23T01:58:35.678044+0000","last_clean":"2024-04-23T01:58:35.678044+0000","last_became_active":"2024-04-23T01:57:51.728785+0000","last_became_peered":"2024-04-23T01:57:51.728785+0000","last_unstale":"2024-04-23T01:58:35.678044+0000","last_undegraded":"2024-04-23T01:58:35.678044+0000","last_fullsized":"2024-04-23T01:58:35.678044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":3008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":208,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:17.527781+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:58:17.527781+0000","last_peered":"2024-04-23T01:58:17.527781+0000","last_clean":"2024-04-23T01:58:17.527781+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:58:17.527781+0000","last_undegraded":"2024-04-23T01:58:17.527781+0000","last_fullsized":"2024-04-23T01:58:17.527781+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"88'295","reported_seq":318,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:36.547575+0000","last_change":"2024-04-23T01:57:51.656207+0000","last_active":"2024-04-23T01:58:36.547575+0000","last_peered":"2024-04-23T01:58:36.547575+0000","last_clean":"2024-04-23T01:58:36.547575+0000","last_became_active":"2024-04-23T01:57:51.638689+0000","last_became_peered":"2024-04-23T01:57:51.638689+0000","last_unstale":"2024-04-23T01:58:36.547575+0000","last_undegraded":"2024-04-23T01:58:36.547575+0000","last_fullsized":"2024-04-23T01:58:36.547575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2424832,"num_objects":37,"num_object_clones":0,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":296,"num_write_kb":2368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":210,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:17.794113+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:58:17.794113+0000","last_peered":"2024-04-23T01:58:17.794113+0000","last_clean":"2024-04-23T01:58:17.794113+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:58:17.794113+0000","last_undegraded":"2024-04-23T01:58:17.794113+0000","last_fullsized":"2024-04-23T01:58:17.794113+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":51183616,"num_objects":784,"num_object_clones":0,"num_object_copies":2352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6248,"num_write_kb":49984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":76783616,"data_stored":153567232,"data_compressed":7926228,"data_compressed_allocated":76783616,"data_compressed_original":153567232,"omap_allocated":0,"internal_metadata":0},"log_size":6233,"ondisk_log_size":6233,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706473,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":140432,"kb_used_data":15048,"kb_used_omap":0,"kb_used_meta":125376,"kb_avail":94231408,"statfs":{"total":96636764160,"available":96492961792,"internally_reserved":0,"allocated":15409152,"data_stored":30044054,"data_compressed":1535660,"data_compressed_allocated":14626816,"data_compressed_original":29745152,"omap_allocated":0,"internal_metadata":128385024},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706473,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":133628,"kb_used_data":22132,"kb_used_omap":0,"kb_used_meta":111488,"kb_avail":94238212,"statfs":{"total":96636764160,"available":96499929088,"internally_reserved":0,"allocated":22663168,"data_stored":44565225,"data_compressed":2285245,"data_compressed_allocated":21889024,"data_compressed_original":44269568,"omap_allocated":0,"internal_metadata":114163712},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706472,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":148196,"kb_used_data":18652,"kb_used_omap":0,"kb_used_meta":129536,"kb_avail":94223644,"statfs":{"total":96636764160,"available":96485011456,"internally_reserved":0,"allocated":19099648,"data_stored":36970697,"data_compressed":1893731,"data_compressed_allocated":18345984,"data_compressed_original":36691968,"omap_allocated":0,"internal_metadata":132644864},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706472,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":146696,"kb_used_data":21312,"kb_used_omap":0,"kb_used_meta":125376,"kb_avail":94225144,"statfs":{"total":96636764160,"available":96486547456,"internally_reserved":0,"allocated":21823488,"data_stored":42885865,"data_compressed":2198662,"data_compressed_allocated":21049344,"data_compressed_original":42590208,"omap_allocated":0,"internal_metadata":128385024},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":21438464,"data_stored":42876928,"data_compressed":2213124,"data_compressed_allocated":21438464,"data_compressed_original":42876928,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":18743296,"data_stored":37486592,"data_compressed":1934762,"data_compressed_allocated":18743296,"data_compressed_original":37486592,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":22007808,"data_stored":44015616,"data_compressed":2271789,"data_compressed_allocated":22007808,"data_compressed_original":44015616,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":14594048,"data_stored":29188096,"data_compressed":1506553,"data_compressed_allocated":14594048,"data_compressed_original":29188096,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:58:39.624 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:58:39.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:39.749+0000 7f6d4c62d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:39.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:39.760+0000 7f6d4c62d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:40.110 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:58:40.110 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:58:40.130 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":492,"stamp":"2024-04-23T01:58:39.707394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":51773984,"num_objects":786,"num_object_clones":0,"num_object_copies":2358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":6385,"num_write_kb":51878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6330,"ondisk_log_size":6330,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":568952,"kb_used_data":77144,"kb_used_omap":0,"kb_used_meta":491776,"kb_avail":376918408,"statfs":{"total":386547056640,"available":385964449792,"internally_reserved":0,"allocated":78995456,"data_stored":154465841,"data_compressed":7913298,"data_compressed_allocated":75911168,"data_compressed_original":153296896,"omap_allocated":0,"internal_metadata":503578624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5480448,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":669,"num_write_kb":5352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000680"},"pg_stats":[{"pgid":"10.7","version":"88'415","reported_seq":440,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:36.601627+0000","last_change":"2024-04-23T01:57:51.583793+0000","last_active":"2024-04-23T01:58:36.601627+0000","last_peered":"2024-04-23T01:58:36.601627+0000","last_clean":"2024-04-23T01:58:36.601627+0000","last_became_active":"2024-04-23T01:57:51.574869+0000","last_became_peered":"2024-04-23T01:57:51.574869+0000","last_unstale":"2024-04-23T01:58:36.601627+0000","last_undegraded":"2024-04-23T01:58:36.601627+0000","last_fullsized":"2024-04-23T01:58:36.601627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3407872,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":3328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"88'407","reported_seq":430,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:36.150983+0000","last_change":"2024-04-23T01:57:51.679265+0000","last_active":"2024-04-23T01:58:36.150983+0000","last_peered":"2024-04-23T01:58:36.150983+0000","last_clean":"2024-04-23T01:58:36.150983+0000","last_became_active":"2024-04-23T01:57:51.662859+0000","last_became_peered":"2024-04-23T01:57:51.662859+0000","last_unstale":"2024-04-23T01:58:36.150983+0000","last_undegraded":"2024-04-23T01:58:36.150983+0000","last_fullsized":"2024-04-23T01:58:36.150983+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342336,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":408,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"88'351","reported_seq":374,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:36.639614+0000","last_change":"2024-04-23T01:57:51.593118+0000","last_active":"2024-04-23T01:58:36.639614+0000","last_peered":"2024-04-23T01:58:36.639614+0000","last_clean":"2024-04-23T01:58:36.639614+0000","last_became_active":"2024-04-23T01:57:51.580553+0000","last_became_peered":"2024-04-23T01:57:51.580553+0000","last_unstale":"2024-04-23T01:58:36.639614+0000","last_undegraded":"2024-04-23T01:58:36.639614+0000","last_fullsized":"2024-04-23T01:58:36.639614+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883584,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":352,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"88'328","reported_seq":355,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T01:58:36.306137+0000","last_change":"2024-04-23T01:57:51.577069+0000","last_active":"2024-04-23T01:58:36.306137+0000","last_peered":"2024-04-23T01:58:36.306137+0000","last_clean":"2024-04-23T01:58:36.306137+0000","last_became_active":"2024-04-23T01:57:51.568227+0000","last_became_peered":"2024-04-23T01:57:51.568227+0000","last_unstale":"2024-04-23T01:58:36.306137+0000","last_undegraded":"2024-04-23T01:58:36.306137+0000","last_fullsized":"2024-04-23T01:58:36.306137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2686976,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":328,"num_write_kb":2624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"88'468","reported_seq":493,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:36.655084+0000","last_change":"2024-04-23T01:57:51.748592+0000","last_active":"2024-04-23T01:58:36.655084+0000","last_peered":"2024-04-23T01:58:36.655084+0000","last_clean":"2024-04-23T01:58:36.655084+0000","last_became_active":"2024-04-23T01:57:51.742159+0000","last_became_peered":"2024-04-23T01:57:51.742159+0000","last_unstale":"2024-04-23T01:58:36.655084+0000","last_undegraded":"2024-04-23T01:58:36.655084+0000","last_fullsized":"2024-04-23T01:58:36.655084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842048,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":469,"num_write_kb":3752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"89'407","reported_seq":435,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T01:58:36.400059+0000","last_change":"2024-04-23T01:57:51.630095+0000","last_active":"2024-04-23T01:58:36.400059+0000","last_peered":"2024-04-23T01:58:36.400059+0000","last_clean":"2024-04-23T01:58:36.400059+0000","last_became_active":"2024-04-23T01:57:51.623783+0000","last_became_peered":"2024-04-23T01:57:51.623783+0000","last_unstale":"2024-04-23T01:58:36.400059+0000","last_undegraded":"2024-04-23T01:58:36.400059+0000","last_fullsized":"2024-04-23T01:58:36.400059+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342336,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":408,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"88'327","reported_seq":352,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:36.382514+0000","last_change":"2024-04-23T01:57:51.605374+0000","last_active":"2024-04-23T01:58:36.382514+0000","last_peered":"2024-04-23T01:58:36.382514+0000","last_clean":"2024-04-23T01:58:36.382514+0000","last_became_active":"2024-04-23T01:57:51.597053+0000","last_became_peered":"2024-04-23T01:57:51.597053+0000","last_unstale":"2024-04-23T01:58:36.382514+0000","last_undegraded":"2024-04-23T01:58:36.382514+0000","last_fullsized":"2024-04-23T01:58:36.382514+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2686976,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":328,"num_write_kb":2624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"88'530","reported_seq":553,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:36.683830+0000","last_change":"2024-04-23T01:57:51.641687+0000","last_active":"2024-04-23T01:58:36.683830+0000","last_peered":"2024-04-23T01:58:36.683830+0000","last_clean":"2024-04-23T01:58:36.683830+0000","last_became_active":"2024-04-23T01:57:51.623973+0000","last_became_peered":"2024-04-23T01:57:51.623973+0000","last_unstale":"2024-04-23T01:58:36.683830+0000","last_undegraded":"2024-04-23T01:58:36.683830+0000","last_fullsized":"2024-04-23T01:58:36.683830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4349952,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":531,"num_write_kb":4248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"88'327","reported_seq":350,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:35.623029+0000","last_change":"2024-04-23T01:57:51.669918+0000","last_active":"2024-04-23T01:58:35.623029+0000","last_peered":"2024-04-23T01:58:35.623029+0000","last_clean":"2024-04-23T01:58:35.623029+0000","last_became_active":"2024-04-23T01:57:51.652610+0000","last_became_peered":"2024-04-23T01:57:51.652610+0000","last_unstale":"2024-04-23T01:58:35.623029+0000","last_undegraded":"2024-04-23T01:58:35.623029+0000","last_fullsized":"2024-04-23T01:58:35.623029+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2686976,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":328,"num_write_kb":2624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":346,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T01:58:36.304591+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:58:36.304591+0000","last_peered":"2024-04-23T01:58:36.304591+0000","last_clean":"2024-04-23T01:58:36.304591+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:58:36.304591+0000","last_undegraded":"2024-04-23T01:58:36.304591+0000","last_fullsized":"2024-04-23T01:58:36.304591+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":210,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:17.792601+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:58:17.792601+0000","last_peered":"2024-04-23T01:58:17.792601+0000","last_clean":"2024-04-23T01:58:17.792601+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:58:17.792601+0000","last_undegraded":"2024-04-23T01:58:17.792601+0000","last_fullsized":"2024-04-23T01:58:17.792601+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"88'416","reported_seq":439,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:36.000916+0000","last_change":"2024-04-23T01:57:51.587471+0000","last_active":"2024-04-23T01:58:36.000916+0000","last_peered":"2024-04-23T01:58:36.000916+0000","last_clean":"2024-04-23T01:58:36.000916+0000","last_became_active":"2024-04-23T01:57:51.575352+0000","last_became_peered":"2024-04-23T01:57:51.575352+0000","last_unstale":"2024-04-23T01:58:36.000916+0000","last_undegraded":"2024-04-23T01:58:36.000916+0000","last_fullsized":"2024-04-23T01:58:36.000916+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3416064,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":417,"num_write_kb":3336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":204,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:17.809011+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:58:17.809011+0000","last_peered":"2024-04-23T01:58:17.809011+0000","last_clean":"2024-04-23T01:58:17.809011+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:58:17.809011+0000","last_undegraded":"2024-04-23T01:58:17.809011+0000","last_fullsized":"2024-04-23T01:58:17.809011+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"89'474","reported_seq":502,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T01:58:36.728139+0000","last_change":"2024-04-23T01:57:51.596134+0000","last_active":"2024-04-23T01:58:36.728139+0000","last_peered":"2024-04-23T01:58:36.728139+0000","last_clean":"2024-04-23T01:58:36.728139+0000","last_became_active":"2024-04-23T01:57:51.589605+0000","last_became_peered":"2024-04-23T01:57:51.589605+0000","last_unstale":"2024-04-23T01:58:36.728139+0000","last_undegraded":"2024-04-23T01:58:36.728139+0000","last_fullsized":"2024-04-23T01:58:36.728139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3891200,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":475,"num_write_kb":3800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":210,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:17.788210+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:58:17.788210+0000","last_peered":"2024-04-23T01:58:17.788210+0000","last_clean":"2024-04-23T01:58:17.788210+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:58:17.788210+0000","last_undegraded":"2024-04-23T01:58:17.788210+0000","last_fullsized":"2024-04-23T01:58:17.788210+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"88'351","reported_seq":376,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:36.121564+0000","last_change":"2024-04-23T01:57:51.662237+0000","last_active":"2024-04-23T01:58:36.121564+0000","last_peered":"2024-04-23T01:58:36.121564+0000","last_clean":"2024-04-23T01:58:36.121564+0000","last_became_active":"2024-04-23T01:57:51.647583+0000","last_became_peered":"2024-04-23T01:57:51.647583+0000","last_unstale":"2024-04-23T01:58:36.121564+0000","last_undegraded":"2024-04-23T01:58:36.121564+0000","last_fullsized":"2024-04-23T01:58:36.121564+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883584,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":352,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":204,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:17.809489+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:58:17.809489+0000","last_peered":"2024-04-23T01:58:17.809489+0000","last_clean":"2024-04-23T01:58:17.809489+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:58:17.809489+0000","last_undegraded":"2024-04-23T01:58:17.809489+0000","last_fullsized":"2024-04-23T01:58:17.809489+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"88'367","reported_seq":392,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:35.391115+0000","last_change":"2024-04-23T01:57:51.681957+0000","last_active":"2024-04-23T01:58:35.391115+0000","last_peered":"2024-04-23T01:58:35.391115+0000","last_clean":"2024-04-23T01:58:35.391115+0000","last_became_active":"2024-04-23T01:57:51.670788+0000","last_became_peered":"2024-04-23T01:57:51.670788+0000","last_unstale":"2024-04-23T01:58:35.391115+0000","last_undegraded":"2024-04-23T01:58:35.391115+0000","last_fullsized":"2024-04-23T01:58:35.391115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3014656,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":368,"num_write_kb":2944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":208,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:17.516673+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:58:17.516673+0000","last_peered":"2024-04-23T01:58:17.516673+0000","last_clean":"2024-04-23T01:58:17.516673+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:58:17.516673+0000","last_undegraded":"2024-04-23T01:58:17.516673+0000","last_fullsized":"2024-04-23T01:58:17.516673+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"88'395","reported_seq":420,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:36.641061+0000","last_change":"2024-04-23T01:57:51.632423+0000","last_active":"2024-04-23T01:58:36.641061+0000","last_peered":"2024-04-23T01:58:36.641061+0000","last_clean":"2024-04-23T01:58:36.641061+0000","last_became_active":"2024-04-23T01:57:51.624354+0000","last_became_peered":"2024-04-23T01:57:51.624354+0000","last_unstale":"2024-04-23T01:58:36.641061+0000","last_undegraded":"2024-04-23T01:58:36.641061+0000","last_fullsized":"2024-04-23T01:58:36.641061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3244032,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":396,"num_write_kb":3168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":213,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T01:58:36.310414+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:58:36.310414+0000","last_peered":"2024-04-23T01:58:36.310414+0000","last_clean":"2024-04-23T01:58:36.310414+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:58:36.310414+0000","last_undegraded":"2024-04-23T01:58:36.310414+0000","last_fullsized":"2024-04-23T01:58:36.310414+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"88'375","reported_seq":400,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:35.678044+0000","last_change":"2024-04-23T01:57:51.735746+0000","last_active":"2024-04-23T01:58:35.678044+0000","last_peered":"2024-04-23T01:58:35.678044+0000","last_clean":"2024-04-23T01:58:35.678044+0000","last_became_active":"2024-04-23T01:57:51.728785+0000","last_became_peered":"2024-04-23T01:57:51.728785+0000","last_unstale":"2024-04-23T01:58:35.678044+0000","last_undegraded":"2024-04-23T01:58:35.678044+0000","last_fullsized":"2024-04-23T01:58:35.678044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":3008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":208,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:17.527781+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:58:17.527781+0000","last_peered":"2024-04-23T01:58:17.527781+0000","last_clean":"2024-04-23T01:58:17.527781+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:58:17.527781+0000","last_undegraded":"2024-04-23T01:58:17.527781+0000","last_fullsized":"2024-04-23T01:58:17.527781+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"88'295","reported_seq":318,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:36.547575+0000","last_change":"2024-04-23T01:57:51.656207+0000","last_active":"2024-04-23T01:58:36.547575+0000","last_peered":"2024-04-23T01:58:36.547575+0000","last_clean":"2024-04-23T01:58:36.547575+0000","last_became_active":"2024-04-23T01:57:51.638689+0000","last_became_peered":"2024-04-23T01:57:51.638689+0000","last_unstale":"2024-04-23T01:58:36.547575+0000","last_undegraded":"2024-04-23T01:58:36.547575+0000","last_fullsized":"2024-04-23T01:58:36.547575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2424832,"num_objects":37,"num_object_clones":0,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":296,"num_write_kb":2368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":210,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T01:58:17.794113+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:58:17.794113+0000","last_peered":"2024-04-23T01:58:17.794113+0000","last_clean":"2024-04-23T01:58:17.794113+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:58:17.794113+0000","last_undegraded":"2024-04-23T01:58:17.794113+0000","last_fullsized":"2024-04-23T01:58:17.794113+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":51183616,"num_objects":784,"num_object_clones":0,"num_object_copies":2352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6248,"num_write_kb":49984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":76783616,"data_stored":153567232,"data_compressed":7926228,"data_compressed_allocated":76783616,"data_compressed_original":153567232,"omap_allocated":0,"internal_metadata":0},"log_size":6233,"ondisk_log_size":6233,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706473,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":140432,"kb_used_data":15048,"kb_used_omap":0,"kb_used_meta":125376,"kb_avail":94231408,"statfs":{"total":96636764160,"available":96492961792,"internally_reserved":0,"allocated":15409152,"data_stored":30044054,"data_compressed":1535660,"data_compressed_allocated":14626816,"data_compressed_original":29745152,"omap_allocated":0,"internal_metadata":128385024},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706473,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":133628,"kb_used_data":22132,"kb_used_omap":0,"kb_used_meta":111488,"kb_avail":94238212,"statfs":{"total":96636764160,"available":96499929088,"internally_reserved":0,"allocated":22663168,"data_stored":44565225,"data_compressed":2285245,"data_compressed_allocated":21889024,"data_compressed_original":44269568,"omap_allocated":0,"internal_metadata":114163712},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706472,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":148196,"kb_used_data":18652,"kb_used_omap":0,"kb_used_meta":129536,"kb_avail":94223644,"statfs":{"total":96636764160,"available":96485011456,"internally_reserved":0,"allocated":19099648,"data_stored":36970697,"data_compressed":1893731,"data_compressed_allocated":18345984,"data_compressed_original":36691968,"omap_allocated":0,"internal_metadata":132644864},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706472,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":146696,"kb_used_data":21312,"kb_used_omap":0,"kb_used_meta":125376,"kb_avail":94225144,"statfs":{"total":96636764160,"available":96486547456,"internally_reserved":0,"allocated":21823488,"data_stored":42885865,"data_compressed":2198662,"data_compressed_allocated":21049344,"data_compressed_original":42590208,"omap_allocated":0,"internal_metadata":128385024},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":21438464,"data_stored":42876928,"data_compressed":2213124,"data_compressed_allocated":21438464,"data_compressed_original":42876928,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":18743296,"data_stored":37486592,"data_compressed":1934762,"data_compressed_allocated":18743296,"data_compressed_original":37486592,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":22007808,"data_stored":44015616,"data_compressed":2271789,"data_compressed_allocated":22007808,"data_compressed_original":44015616,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":14594048,"data_stored":29188096,"data_compressed":1506553,"data_compressed_allocated":14594048,"data_compressed_original":29188096,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:58:40.131 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-23T01:58:40.132 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:58:40.132 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:58:40.132 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:58:40.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:40.255+0000 7fe84fc2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:40.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:40.266+0000 7fe84fc2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:40.369 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:58:40.369 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:58:40.369 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:58:40.369 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:58:40.369 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:58:40.369 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:58:40.369 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:58:40.369 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:58:40.369 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:58:40.369 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:58:40.369 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:58:40.369 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:58:40.372 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:58:40.373 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:58:40.373 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:58:40.485 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 48 16 6749 6733 1.09577 0.992188 0.0705389 0.113883 2024-04-23T01:58:40.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:40.517+0000 7f880ab9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:40.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:40.529+0000 7f880ab9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:40.568 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:58:40.569 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:58:40.569 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:58:40.569 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:58:40.569 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:58:40.569 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:58:40.569 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:58:40.569 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:58:40.569 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:58:40.569 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:58:40.569 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:58:40.569 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:58:40.572 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:58:40.572 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:58:40.572 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:58:40.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:40.707+0000 7fcb526ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:40.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:40.718+0000 7fcb526ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:40.737 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:58:40.737 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:58:40.737 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:58:40.737 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:58:40.737 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:58:40.737 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:58:40.737 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:58:40.737 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:58:40.737 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:58:40.737 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:58:40.738 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:58:40.738 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:58:40.740 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:58:40.741 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:58:40.741 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:58:40.770 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:58:40.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:40.859+0000 7faaf762d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:40.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:40.874+0000 7faaf762d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:40.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:40.901+0000 7ff7c42ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:40.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:40.908+0000 7ff7c42ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:40.962 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:58:40.962 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:58:40.962 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:58:40.963 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:58:40.963 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:58:40.963 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:58:40.963 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:58:40.963 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:58:40.963 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:58:40.963 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:58:40.963 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:58:40.963 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:58:40.965 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:58:40.966 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:58:41.485 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 49 16 6884 6868 1.09493 1.05469 0.112584 0.113985 2024-04-23T01:58:42.485 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 50 16 7021 7005 1.09443 1.07031 0.0442975 0.11406 2024-04-23T01:58:43.121 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:58:43.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:43.240+0000 7fedef39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:43.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:43.246+0000 7fedef39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:43.485 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 51 16 7145 7129 1.09197 0.96875 0.219133 0.114262 2024-04-23T01:58:44.485 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 52 16 7271 7255 1.0899 0.984375 0.278491 0.114425 2024-04-23T01:58:45.368 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:58:45.485 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 53 16 7400 7384 1.08835 1.00781 0.0669214 0.114727 2024-04-23T01:58:45.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:45.487+0000 7f854da2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:45.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:45.492+0000 7f854da2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:45.967 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:58:45.967 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:58:45.967 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:58:45.967 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:58:46.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:46.102+0000 7f301bf5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:46.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:46.117+0000 7f301bf5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:46.180 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:58:46.180 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:58:46.180 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:58:46.180 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:58:46.180 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:58:46.180 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:58:46.180 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:58:46.180 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:58:46.181 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:58:46.181 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:58:46.181 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:58:46.181 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:58:46.183 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:58:46.184 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:58:46.184 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:58:46.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:46.326+0000 7f2403d07640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:46.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:46.338+0000 7f2403d07640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:46.456 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:58:46.457 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:58:46.457 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:58:46.457 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:58:46.457 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:58:46.457 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:58:46.457 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:58:46.457 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:58:46.457 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:58:46.457 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:58:46.457 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:58:46.457 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:58:46.459 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:58:46.460 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:58:46.460 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:58:46.485 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 54 16 7546 7530 1.08931 1.14062 0.0867504 0.114669 2024-04-23T01:58:46.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:46.579+0000 7f741bd9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:46.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:46.589+0000 7f741bd9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:46.607 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:58:46.608 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:58:46.608 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:58:46.608 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:58:46.608 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:58:46.608 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:58:46.608 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:58:46.608 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:58:46.608 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:58:46.608 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:58:46.608 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:58:46.608 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:58:46.611 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:58:46.611 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:58:46.611 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:58:46.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:46.771+0000 7fed1a22d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:46.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:46.783+0000 7fed1a22d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:46.820 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:58:46.821 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:58:46.821 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:58:46.821 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:58:46.821 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:58:46.821 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:58:46.821 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:58:46.821 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:58:46.821 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:58:46.821 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:58:46.821 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:58:46.821 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:58:46.823 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:58:46.824 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:58:47.485 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 55 16 7662 7646 1.08598 0.90625 0.203747 0.114935 2024-04-23T01:58:47.861 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:58:47.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:47.978+0000 7f5e2a06d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:47.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:47.984+0000 7f5e2a06d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:48.485 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 56 16 7797 7781 1.08542 1.05469 0.179039 0.115014 2024-04-23T01:58:49.485 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 57 16 7935 7919 1.08529 1.07812 0.115781 0.115044 2024-04-23T01:58:50.264 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:58:50.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:50.415+0000 7f3b1ff5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:50.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:50.422+0000 7f3b1ff5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:50.486 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 58 16 8075 8059 1.08544 1.09375 0.203532 0.115035 2024-04-23T01:58:51.486 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 59 16 8200 8184 1.08359 0.976562 0.123011 0.115215 2024-04-23T01:58:51.824 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:58:51.825 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:58:51.825 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:58:51.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:51.936+0000 7fe3d8f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:51.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:51.946+0000 7fe3d8f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:52.153 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:58:52.154 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:58:52.167 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":498,"stamp":"2024-04-23T01:58:51.709992+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":62448160,"num_objects":948,"num_object_clones":0,"num_object_copies":2844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":7688,"num_write_kb":62302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7632,"ondisk_log_size":7632,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":584608,"kb_used_data":92800,"kb_used_omap":0,"kb_used_meta":491776,"kb_avail":376902752,"statfs":{"total":386547056640,"available":385948418048,"internally_reserved":0,"allocated":95027200,"data_stored":186332244,"data_compressed":9556230,"data_compressed_allocated":91820032,"data_compressed_original":185114624,"omap_allocated":0,"internal_metadata":503578624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10674176,"num_objects":162,"num_object_clones":0,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1303,"num_write_kb":10424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002597"},"pg_stats":[{"pgid":"10.7","version":"92'495","reported_seq":532,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:45.897744+0000","last_change":"2024-04-23T01:57:51.583793+0000","last_active":"2024-04-23T01:58:45.897744+0000","last_peered":"2024-04-23T01:58:45.897744+0000","last_clean":"2024-04-23T01:58:45.897744+0000","last_became_active":"2024-04-23T01:57:51.574869+0000","last_became_peered":"2024-04-23T01:57:51.574869+0000","last_unstale":"2024-04-23T01:58:45.897744+0000","last_undegraded":"2024-04-23T01:58:45.897744+0000","last_fullsized":"2024-04-23T01:58:45.897744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4063232,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":496,"num_write_kb":3968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"92'465","reported_seq":496,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:46.683478+0000","last_change":"2024-04-23T01:57:51.679265+0000","last_active":"2024-04-23T01:58:46.683478+0000","last_peered":"2024-04-23T01:58:46.683478+0000","last_clean":"2024-04-23T01:58:46.683478+0000","last_became_active":"2024-04-23T01:57:51.662859+0000","last_became_peered":"2024-04-23T01:57:51.662859+0000","last_unstale":"2024-04-23T01:58:46.683478+0000","last_undegraded":"2024-04-23T01:58:46.683478+0000","last_fullsized":"2024-04-23T01:58:46.683478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3817472,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":466,"num_write_kb":3728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"92'423","reported_seq":456,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:45.760894+0000","last_change":"2024-04-23T01:57:51.593118+0000","last_active":"2024-04-23T01:58:45.760894+0000","last_peered":"2024-04-23T01:58:45.760894+0000","last_clean":"2024-04-23T01:58:45.760894+0000","last_became_active":"2024-04-23T01:57:51.580553+0000","last_became_peered":"2024-04-23T01:57:51.580553+0000","last_unstale":"2024-04-23T01:58:45.760894+0000","last_undegraded":"2024-04-23T01:58:45.760894+0000","last_fullsized":"2024-04-23T01:58:45.760894+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3473408,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":424,"num_write_kb":3392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"92'455","reported_seq":490,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:46.374215+0000","last_change":"2024-04-23T01:57:51.577069+0000","last_active":"2024-04-23T01:58:46.374215+0000","last_peered":"2024-04-23T01:58:46.374215+0000","last_clean":"2024-04-23T01:58:46.374215+0000","last_became_active":"2024-04-23T01:57:51.568227+0000","last_became_peered":"2024-04-23T01:57:51.568227+0000","last_unstale":"2024-04-23T01:58:46.374215+0000","last_undegraded":"2024-04-23T01:58:46.374215+0000","last_fullsized":"2024-04-23T01:58:46.374215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":456,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"92'543","reported_seq":580,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:46.064756+0000","last_change":"2024-04-23T01:57:51.748592+0000","last_active":"2024-04-23T01:58:46.064756+0000","last_peered":"2024-04-23T01:58:46.064756+0000","last_clean":"2024-04-23T01:58:46.064756+0000","last_became_active":"2024-04-23T01:57:51.742159+0000","last_became_peered":"2024-04-23T01:57:51.742159+0000","last_unstale":"2024-04-23T01:58:46.064756+0000","last_undegraded":"2024-04-23T01:58:46.064756+0000","last_fullsized":"2024-04-23T01:58:46.064756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":544,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"92'488","reported_seq":523,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:46.328428+0000","last_change":"2024-04-23T01:57:51.630095+0000","last_active":"2024-04-23T01:58:46.328428+0000","last_peered":"2024-04-23T01:58:46.328428+0000","last_clean":"2024-04-23T01:58:46.328428+0000","last_became_active":"2024-04-23T01:57:51.623783+0000","last_became_peered":"2024-04-23T01:57:51.623783+0000","last_unstale":"2024-04-23T01:58:46.328428+0000","last_undegraded":"2024-04-23T01:58:46.328428+0000","last_fullsized":"2024-04-23T01:58:46.328428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4005888,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":489,"num_write_kb":3912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"92'439","reported_seq":476,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:46.653767+0000","last_change":"2024-04-23T01:57:51.605374+0000","last_active":"2024-04-23T01:58:46.653767+0000","last_peered":"2024-04-23T01:58:46.653767+0000","last_clean":"2024-04-23T01:58:46.653767+0000","last_became_active":"2024-04-23T01:57:51.597053+0000","last_became_peered":"2024-04-23T01:57:51.597053+0000","last_unstale":"2024-04-23T01:58:46.653767+0000","last_undegraded":"2024-04-23T01:58:46.653767+0000","last_fullsized":"2024-04-23T01:58:46.653767+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3604480,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":440,"num_write_kb":3520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"92'599","reported_seq":630,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:46.054822+0000","last_change":"2024-04-23T01:57:51.641687+0000","last_active":"2024-04-23T01:58:46.054822+0000","last_peered":"2024-04-23T01:58:46.054822+0000","last_clean":"2024-04-23T01:58:46.054822+0000","last_became_active":"2024-04-23T01:57:51.623973+0000","last_became_peered":"2024-04-23T01:57:51.623973+0000","last_unstale":"2024-04-23T01:58:46.054822+0000","last_undegraded":"2024-04-23T01:58:46.054822+0000","last_fullsized":"2024-04-23T01:58:46.054822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"92'415","reported_seq":446,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:46.443457+0000","last_change":"2024-04-23T01:57:51.669918+0000","last_active":"2024-04-23T01:58:46.443457+0000","last_peered":"2024-04-23T01:58:46.443457+0000","last_clean":"2024-04-23T01:58:46.443457+0000","last_became_active":"2024-04-23T01:57:51.652610+0000","last_became_peered":"2024-04-23T01:57:51.652610+0000","last_unstale":"2024-04-23T01:58:46.443457+0000","last_undegraded":"2024-04-23T01:58:46.443457+0000","last_fullsized":"2024-04-23T01:58:46.443457+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3407872,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":3328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":353,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.603118+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:58:39.603118+0000","last_peered":"2024-04-23T01:58:39.603118+0000","last_clean":"2024-04-23T01:58:39.603118+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:58:39.603118+0000","last_undegraded":"2024-04-23T01:58:39.603118+0000","last_fullsized":"2024-04-23T01:58:39.603118+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":218,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.642652+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:58:39.642652+0000","last_peered":"2024-04-23T01:58:39.642652+0000","last_clean":"2024-04-23T01:58:39.642652+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:58:39.642652+0000","last_undegraded":"2024-04-23T01:58:39.642652+0000","last_fullsized":"2024-04-23T01:58:39.642652+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"92'543","reported_seq":576,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:46.466156+0000","last_change":"2024-04-23T01:57:51.587471+0000","last_active":"2024-04-23T01:58:46.466156+0000","last_peered":"2024-04-23T01:58:46.466156+0000","last_clean":"2024-04-23T01:58:46.466156+0000","last_became_active":"2024-04-23T01:57:51.575352+0000","last_became_peered":"2024-04-23T01:57:51.575352+0000","last_unstale":"2024-04-23T01:58:46.466156+0000","last_undegraded":"2024-04-23T01:58:46.466156+0000","last_fullsized":"2024-04-23T01:58:46.466156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":544,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":216,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.610638+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:58:39.610638+0000","last_peered":"2024-04-23T01:58:39.610638+0000","last_clean":"2024-04-23T01:58:39.610638+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:58:39.610638+0000","last_undegraded":"2024-04-23T01:58:39.610638+0000","last_fullsized":"2024-04-23T01:58:39.610638+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"92'519","reported_seq":554,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:45.869665+0000","last_change":"2024-04-23T01:57:51.596134+0000","last_active":"2024-04-23T01:58:45.869665+0000","last_peered":"2024-04-23T01:58:45.869665+0000","last_clean":"2024-04-23T01:58:45.869665+0000","last_became_active":"2024-04-23T01:57:51.589605+0000","last_became_peered":"2024-04-23T01:57:51.589605+0000","last_unstale":"2024-04-23T01:58:45.869665+0000","last_undegraded":"2024-04-23T01:58:45.869665+0000","last_fullsized":"2024-04-23T01:58:45.869665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259840,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":520,"num_write_kb":4160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":218,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.617202+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:58:39.617202+0000","last_peered":"2024-04-23T01:58:39.617202+0000","last_clean":"2024-04-23T01:58:39.617202+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:58:39.617202+0000","last_undegraded":"2024-04-23T01:58:39.617202+0000","last_fullsized":"2024-04-23T01:58:39.617202+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"92'463","reported_seq":500,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:45.626827+0000","last_change":"2024-04-23T01:57:51.662237+0000","last_active":"2024-04-23T01:58:45.626827+0000","last_peered":"2024-04-23T01:58:45.626827+0000","last_clean":"2024-04-23T01:58:45.626827+0000","last_became_active":"2024-04-23T01:57:51.647583+0000","last_became_peered":"2024-04-23T01:57:51.647583+0000","last_unstale":"2024-04-23T01:58:45.626827+0000","last_undegraded":"2024-04-23T01:58:45.626827+0000","last_fullsized":"2024-04-23T01:58:45.626827+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":464,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":216,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.622044+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:58:39.622044+0000","last_peered":"2024-04-23T01:58:39.622044+0000","last_clean":"2024-04-23T01:58:39.622044+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:58:39.622044+0000","last_undegraded":"2024-04-23T01:58:39.622044+0000","last_fullsized":"2024-04-23T01:58:39.622044+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"92'463","reported_seq":500,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:45.539078+0000","last_change":"2024-04-23T01:57:51.681957+0000","last_active":"2024-04-23T01:58:45.539078+0000","last_peered":"2024-04-23T01:58:45.539078+0000","last_clean":"2024-04-23T01:58:45.539078+0000","last_became_active":"2024-04-23T01:57:51.670788+0000","last_became_peered":"2024-04-23T01:57:51.670788+0000","last_unstale":"2024-04-23T01:58:45.539078+0000","last_undegraded":"2024-04-23T01:58:45.539078+0000","last_fullsized":"2024-04-23T01:58:45.539078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":464,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":218,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.283014+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:58:39.283014+0000","last_peered":"2024-04-23T01:58:39.283014+0000","last_clean":"2024-04-23T01:58:39.283014+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:58:39.283014+0000","last_undegraded":"2024-04-23T01:58:39.283014+0000","last_fullsized":"2024-04-23T01:58:39.283014+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"92'459","reported_seq":496,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:46.668690+0000","last_change":"2024-04-23T01:57:51.632423+0000","last_active":"2024-04-23T01:58:46.668690+0000","last_peered":"2024-04-23T01:58:46.668690+0000","last_clean":"2024-04-23T01:58:46.668690+0000","last_became_active":"2024-04-23T01:57:51.624354+0000","last_became_peered":"2024-04-23T01:57:51.624354+0000","last_unstale":"2024-04-23T01:58:46.668690+0000","last_undegraded":"2024-04-23T01:58:46.668690+0000","last_fullsized":"2024-04-23T01:58:46.668690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3768320,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":460,"num_write_kb":3680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":220,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.604585+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:58:39.604585+0000","last_peered":"2024-04-23T01:58:39.604585+0000","last_clean":"2024-04-23T01:58:39.604585+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:58:39.604585+0000","last_undegraded":"2024-04-23T01:58:39.604585+0000","last_fullsized":"2024-04-23T01:58:39.604585+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"92'415","reported_seq":452,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:44.109140+0000","last_change":"2024-04-23T01:57:51.735746+0000","last_active":"2024-04-23T01:58:44.109140+0000","last_peered":"2024-04-23T01:58:44.109140+0000","last_clean":"2024-04-23T01:58:44.109140+0000","last_became_active":"2024-04-23T01:57:51.728785+0000","last_became_peered":"2024-04-23T01:57:51.728785+0000","last_unstale":"2024-04-23T01:58:44.109140+0000","last_undegraded":"2024-04-23T01:58:44.109140+0000","last_fullsized":"2024-04-23T01:58:44.109140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3407872,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":3328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":218,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.283528+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:58:39.283528+0000","last_peered":"2024-04-23T01:58:39.283528+0000","last_clean":"2024-04-23T01:58:39.283528+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:58:39.283528+0000","last_undegraded":"2024-04-23T01:58:39.283528+0000","last_fullsized":"2024-04-23T01:58:39.283528+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"92'351","reported_seq":382,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:44.879453+0000","last_change":"2024-04-23T01:57:51.656207+0000","last_active":"2024-04-23T01:58:44.879453+0000","last_peered":"2024-04-23T01:58:44.879453+0000","last_clean":"2024-04-23T01:58:44.879453+0000","last_became_active":"2024-04-23T01:57:51.638689+0000","last_became_peered":"2024-04-23T01:57:51.638689+0000","last_unstale":"2024-04-23T01:58:44.879453+0000","last_undegraded":"2024-04-23T01:58:44.879453+0000","last_fullsized":"2024-04-23T01:58:44.879453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883584,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":352,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":218,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.624179+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:58:39.624179+0000","last_peered":"2024-04-23T01:58:39.624179+0000","last_clean":"2024-04-23T01:58:39.624179+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:58:39.624179+0000","last_undegraded":"2024-04-23T01:58:39.624179+0000","last_fullsized":"2024-04-23T01:58:39.624179+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":61857792,"num_objects":946,"num_object_clones":0,"num_object_copies":2838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7551,"num_write_kb":60408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":92811264,"data_stored":185622528,"data_compressed":9581427,"data_compressed_allocated":92811264,"data_compressed_original":185622528,"omap_allocated":0,"internal_metadata":0},"log_size":7535,"ondisk_log_size":7535,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":147520,"kb_used_data":17976,"kb_used_omap":0,"kb_used_meta":129536,"kb_avail":94224320,"statfs":{"total":96636764160,"available":96485703680,"internally_reserved":0,"allocated":18407424,"data_stored":36001181,"data_compressed":1842758,"data_compressed_allocated":17600512,"data_compressed_original":35692544,"omap_allocated":0,"internal_metadata":132644864},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706483,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":125952,"kb_used_data":26936,"kb_used_omap":0,"kb_used_meta":99008,"kb_avail":94245888,"statfs":{"total":96636764160,"available":96507789312,"internally_reserved":0,"allocated":27582464,"data_stored":54351261,"data_compressed":2789884,"data_compressed_allocated":26775552,"data_compressed_original":54042624,"omap_allocated":0,"internal_metadata":101384192},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":156116,"kb_used_data":22412,"kb_used_omap":0,"kb_used_meta":133696,"kb_avail":94215724,"statfs":{"total":96636764160,"available":96476901376,"internally_reserved":0,"allocated":22949888,"data_stored":44618621,"data_compressed":2287967,"data_compressed_allocated":22163456,"data_compressed_original":44326912,"omap_allocated":0,"internal_metadata":136904704},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155020,"kb_used_data":25476,"kb_used_omap":0,"kb_used_meta":129536,"kb_avail":94216820,"statfs":{"total":96636764160,"available":96478023680,"internally_reserved":0,"allocated":26087424,"data_stored":51361181,"data_compressed":2635621,"data_compressed_allocated":25280512,"data_compressed_original":51052544,"omap_allocated":0,"internal_metadata":132644864},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":25608192,"data_stored":51216384,"data_compressed":2643738,"data_compressed_allocated":25608192,"data_compressed_original":51216384,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22536192,"data_stored":45072384,"data_compressed":2326460,"data_compressed_allocated":22536192,"data_compressed_original":45072384,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":26828800,"data_stored":53657600,"data_compressed":2769660,"data_compressed_allocated":26828800,"data_compressed_original":53657600,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":17838080,"data_stored":35676160,"data_compressed":1841569,"data_compressed_allocated":17838080,"data_compressed_original":35676160,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:58:52.486 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:58:52.485153+0000 min lat: 0.0138177 max lat: 0.388479 avg lat: 0.115258 2024-04-23T01:58:52.486 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:58:52.486 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 60 16 8334 8318 1.08298 1.04688 0.105377 0.115258 2024-04-23T01:58:52.654 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:58:52.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:52.810+0000 7fecb139d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:52.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:52.816+0000 7fecb139d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:53.486 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 61 16 8457 8441 1.08097 0.960938 0.237026 0.115495 2024-04-23T01:58:54.486 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 62 16 8594 8578 1.0808 1.07031 0.161867 0.11551 2024-04-23T01:58:54.952 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:58:55.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:55.062+0000 7fbe43c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:55.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:55.068+0000 7fbe43c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:55.486 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 63 16 8709 8693 1.07791 0.898438 0.155547 0.115814 2024-04-23T01:58:56.486 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 64 16 8833 8817 1.0762 0.96875 0.129759 0.115973 2024-04-23T01:58:57.170 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:58:57.170 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:58:57.170 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:58:57.170 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:58:57.228 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:58:57.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:57.283+0000 7f888455d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:57.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:57.295+0000 7f888455d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:57.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:57.347+0000 7f777d0ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:57.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:57.354+0000 7f777d0ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:57.433 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:58:57.433 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:58:57.433 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:58:57.433 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:58:57.433 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:58:57.434 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:58:57.434 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:58:57.434 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:58:57.434 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:58:57.434 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:58:57.434 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:58:57.434 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:58:57.437 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:58:57.437 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:58:57.437 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:58:57.486 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 65 16 8957 8941 1.07454 0.96875 0.143053 0.116157 2024-04-23T01:58:57.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:57.565+0000 7f34bac64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:57.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:57.578+0000 7f34bac64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:57.628 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:58:57.628 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:58:57.628 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:58:57.629 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:58:57.629 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:58:57.629 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:58:57.629 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:58:57.629 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:58:57.629 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:58:57.629 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:58:57.629 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:58:57.629 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:58:57.631 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:58:57.631 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:58:57.632 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:58:57.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:57.794+0000 7f0904c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:57.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:57.807+0000 7f0904c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:57.843 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:58:57.843 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:58:57.843 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:58:57.843 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:58:57.843 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:58:57.843 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:58:57.843 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:58:57.843 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:58:57.843 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:58:57.843 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:58:57.843 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:58:57.843 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:58:57.845 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:58:57.845 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:58:57.846 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:58:57.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:57.981+0000 7f0933e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:57.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:57.993+0000 7f0933e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:58.037 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:58:58.038 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:58:58.038 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:58:58.038 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:58:58.038 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:58:58.038 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:58:58.038 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:58:58.038 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:58:58.038 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:58:58.038 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:58:58.038 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:58:58.038 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:58:58.041 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:58:58.041 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:58:58.486 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 66 16 9088 9072 1.07377 1.02344 0.092859 0.116297 2024-04-23T01:58:59.486 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 67 16 9201 9185 1.07092 0.882812 0.0612944 0.116515 2024-04-23T01:58:59.598 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:58:59.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:59.740+0000 7fcd1fc6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:58:59.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:58:59.748+0000 7fcd1fc6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:00.486 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 68 16 9343 9327 1.07148 1.10938 0.0444031 0.116549 2024-04-23T01:59:01.487 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 69 16 9474 9458 1.07078 1.02344 0.182152 0.116568 2024-04-23T01:59:01.966 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:59:02.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:02.086+0000 7fe2b8a17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:02.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:02.091+0000 7fe2b8a17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:02.487 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 70 16 9589 9573 1.06832 0.898438 0.118848 0.116861 2024-04-23T01:59:03.042 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:59:03.042 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:59:03.043 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:59:03.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:03.159+0000 7f362a35d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:03.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:03.168+0000 7f362a35d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:03.386 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:59:03.387 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:59:03.401 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":503,"stamp":"2024-04-23T01:59:01.711858+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":73114144,"num_objects":1110,"num_object_clones":0,"num_object_copies":3330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":8990,"num_write_kb":72718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8934,"ondisk_log_size":8934,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":579444,"kb_used_data":108432,"kb_used_omap":0,"kb_used_meta":470976,"kb_avail":376907916,"statfs":{"total":386547056640,"available":385953705984,"internally_reserved":0,"allocated":111034368,"data_stored":218346580,"data_compressed":11209314,"data_compressed_allocated":107827200,"data_compressed_original":217128960,"omap_allocated":0,"internal_metadata":482279424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10665984,"num_objects":162,"num_object_clones":0,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1302,"num_write_kb":10416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002291"},"pg_stats":[{"pgid":"10.7","version":"92'559","reported_seq":596,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:55.556774+0000","last_change":"2024-04-23T01:57:51.583793+0000","last_active":"2024-04-23T01:58:55.556774+0000","last_peered":"2024-04-23T01:58:55.556774+0000","last_clean":"2024-04-23T01:58:55.556774+0000","last_became_active":"2024-04-23T01:57:51.574869+0000","last_became_peered":"2024-04-23T01:57:51.574869+0000","last_unstale":"2024-04-23T01:58:55.556774+0000","last_undegraded":"2024-04-23T01:58:55.556774+0000","last_fullsized":"2024-04-23T01:58:55.556774+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4587520,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":560,"num_write_kb":4480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"92'574","reported_seq":605,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:56.692657+0000","last_change":"2024-04-23T01:57:51.679265+0000","last_active":"2024-04-23T01:58:56.692657+0000","last_peered":"2024-04-23T01:58:56.692657+0000","last_clean":"2024-04-23T01:58:56.692657+0000","last_became_active":"2024-04-23T01:57:51.662859+0000","last_became_peered":"2024-04-23T01:57:51.662859+0000","last_unstale":"2024-04-23T01:58:56.692657+0000","last_undegraded":"2024-04-23T01:58:56.692657+0000","last_fullsized":"2024-04-23T01:58:56.692657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4710400,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":575,"num_write_kb":4600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"92'482","reported_seq":515,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:56.754629+0000","last_change":"2024-04-23T01:57:51.593118+0000","last_active":"2024-04-23T01:58:56.754629+0000","last_peered":"2024-04-23T01:58:56.754629+0000","last_clean":"2024-04-23T01:58:56.754629+0000","last_became_active":"2024-04-23T01:57:51.580553+0000","last_became_peered":"2024-04-23T01:57:51.580553+0000","last_unstale":"2024-04-23T01:58:56.754629+0000","last_undegraded":"2024-04-23T01:58:56.754629+0000","last_fullsized":"2024-04-23T01:58:56.754629+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3956736,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":483,"num_write_kb":3864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"92'527","reported_seq":562,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:56.274909+0000","last_change":"2024-04-23T01:57:51.577069+0000","last_active":"2024-04-23T01:58:56.274909+0000","last_peered":"2024-04-23T01:58:56.274909+0000","last_clean":"2024-04-23T01:58:56.274909+0000","last_became_active":"2024-04-23T01:57:51.568227+0000","last_became_peered":"2024-04-23T01:57:51.568227+0000","last_unstale":"2024-04-23T01:58:56.274909+0000","last_undegraded":"2024-04-23T01:58:56.274909+0000","last_fullsized":"2024-04-23T01:58:56.274909+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":528,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"92'647","reported_seq":684,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:56.212339+0000","last_change":"2024-04-23T01:57:51.748592+0000","last_active":"2024-04-23T01:58:56.212339+0000","last_peered":"2024-04-23T01:58:56.212339+0000","last_clean":"2024-04-23T01:58:56.212339+0000","last_became_active":"2024-04-23T01:57:51.742159+0000","last_became_peered":"2024-04-23T01:57:51.742159+0000","last_unstale":"2024-04-23T01:58:56.212339+0000","last_undegraded":"2024-04-23T01:58:56.212339+0000","last_fullsized":"2024-04-23T01:58:56.212339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308416,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":648,"num_write_kb":5184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"92'615","reported_seq":650,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:56.070197+0000","last_change":"2024-04-23T01:57:51.630095+0000","last_active":"2024-04-23T01:58:56.070197+0000","last_peered":"2024-04-23T01:58:56.070197+0000","last_clean":"2024-04-23T01:58:56.070197+0000","last_became_active":"2024-04-23T01:57:51.623783+0000","last_became_peered":"2024-04-23T01:57:51.623783+0000","last_unstale":"2024-04-23T01:58:56.070197+0000","last_undegraded":"2024-04-23T01:58:56.070197+0000","last_fullsized":"2024-04-23T01:58:56.070197+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5046272,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":616,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"92'527","reported_seq":564,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:56.548774+0000","last_change":"2024-04-23T01:57:51.605374+0000","last_active":"2024-04-23T01:58:56.548774+0000","last_peered":"2024-04-23T01:58:56.548774+0000","last_clean":"2024-04-23T01:58:56.548774+0000","last_became_active":"2024-04-23T01:57:51.597053+0000","last_became_peered":"2024-04-23T01:57:51.597053+0000","last_unstale":"2024-04-23T01:58:56.548774+0000","last_undegraded":"2024-04-23T01:58:56.548774+0000","last_fullsized":"2024-04-23T01:58:56.548774+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":528,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"92'671","reported_seq":702,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:56.491333+0000","last_change":"2024-04-23T01:57:51.641687+0000","last_active":"2024-04-23T01:58:56.491333+0000","last_peered":"2024-04-23T01:58:56.491333+0000","last_clean":"2024-04-23T01:58:56.491333+0000","last_became_active":"2024-04-23T01:57:51.623973+0000","last_became_peered":"2024-04-23T01:57:51.623973+0000","last_unstale":"2024-04-23T01:58:56.491333+0000","last_undegraded":"2024-04-23T01:58:56.491333+0000","last_fullsized":"2024-04-23T01:58:56.491333+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505024,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":672,"num_write_kb":5376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"92'487","reported_seq":518,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:54.317012+0000","last_change":"2024-04-23T01:57:51.669918+0000","last_active":"2024-04-23T01:58:54.317012+0000","last_peered":"2024-04-23T01:58:54.317012+0000","last_clean":"2024-04-23T01:58:54.317012+0000","last_became_active":"2024-04-23T01:57:51.652610+0000","last_became_peered":"2024-04-23T01:57:51.652610+0000","last_unstale":"2024-04-23T01:58:54.317012+0000","last_undegraded":"2024-04-23T01:58:54.317012+0000","last_fullsized":"2024-04-23T01:58:54.317012+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3997696,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":488,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":353,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.603118+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:58:39.603118+0000","last_peered":"2024-04-23T01:58:39.603118+0000","last_clean":"2024-04-23T01:58:39.603118+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:58:39.603118+0000","last_undegraded":"2024-04-23T01:58:39.603118+0000","last_fullsized":"2024-04-23T01:58:39.603118+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":218,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.642652+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:58:39.642652+0000","last_peered":"2024-04-23T01:58:39.642652+0000","last_clean":"2024-04-23T01:58:39.642652+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:58:39.642652+0000","last_undegraded":"2024-04-23T01:58:39.642652+0000","last_fullsized":"2024-04-23T01:58:39.642652+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"92'631","reported_seq":664,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:56.157138+0000","last_change":"2024-04-23T01:57:51.587471+0000","last_active":"2024-04-23T01:58:56.157138+0000","last_peered":"2024-04-23T01:58:56.157138+0000","last_clean":"2024-04-23T01:58:56.157138+0000","last_became_active":"2024-04-23T01:57:51.575352+0000","last_became_peered":"2024-04-23T01:57:51.575352+0000","last_unstale":"2024-04-23T01:58:56.157138+0000","last_undegraded":"2024-04-23T01:58:56.157138+0000","last_fullsized":"2024-04-23T01:58:56.157138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":5056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":216,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.610638+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:58:39.610638+0000","last_peered":"2024-04-23T01:58:39.610638+0000","last_clean":"2024-04-23T01:58:39.610638+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:58:39.610638+0000","last_undegraded":"2024-04-23T01:58:39.610638+0000","last_fullsized":"2024-04-23T01:58:39.610638+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"92'607","reported_seq":642,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:55.557034+0000","last_change":"2024-04-23T01:57:51.596134+0000","last_active":"2024-04-23T01:58:55.557034+0000","last_peered":"2024-04-23T01:58:55.557034+0000","last_clean":"2024-04-23T01:58:55.557034+0000","last_became_active":"2024-04-23T01:57:51.589605+0000","last_became_peered":"2024-04-23T01:57:51.589605+0000","last_unstale":"2024-04-23T01:58:55.557034+0000","last_undegraded":"2024-04-23T01:58:55.557034+0000","last_fullsized":"2024-04-23T01:58:55.557034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4980736,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":608,"num_write_kb":4864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":218,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.617202+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:58:39.617202+0000","last_peered":"2024-04-23T01:58:39.617202+0000","last_clean":"2024-04-23T01:58:39.617202+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:58:39.617202+0000","last_undegraded":"2024-04-23T01:58:39.617202+0000","last_fullsized":"2024-04-23T01:58:39.617202+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"92'519","reported_seq":556,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:56.360451+0000","last_change":"2024-04-23T01:57:51.662237+0000","last_active":"2024-04-23T01:58:56.360451+0000","last_peered":"2024-04-23T01:58:56.360451+0000","last_clean":"2024-04-23T01:58:56.360451+0000","last_became_active":"2024-04-23T01:57:51.647583+0000","last_became_peered":"2024-04-23T01:57:51.647583+0000","last_unstale":"2024-04-23T01:58:56.360451+0000","last_undegraded":"2024-04-23T01:58:56.360451+0000","last_fullsized":"2024-04-23T01:58:56.360451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259840,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":520,"num_write_kb":4160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":216,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.622044+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:58:39.622044+0000","last_peered":"2024-04-23T01:58:39.622044+0000","last_clean":"2024-04-23T01:58:39.622044+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:58:39.622044+0000","last_undegraded":"2024-04-23T01:58:39.622044+0000","last_fullsized":"2024-04-23T01:58:39.622044+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"92'554","reported_seq":591,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:56.657966+0000","last_change":"2024-04-23T01:57:51.681957+0000","last_active":"2024-04-23T01:58:56.657966+0000","last_peered":"2024-04-23T01:58:56.657966+0000","last_clean":"2024-04-23T01:58:56.657966+0000","last_became_active":"2024-04-23T01:57:51.670788+0000","last_became_peered":"2024-04-23T01:57:51.670788+0000","last_unstale":"2024-04-23T01:58:56.657966+0000","last_undegraded":"2024-04-23T01:58:56.657966+0000","last_fullsized":"2024-04-23T01:58:56.657966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4546560,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":555,"num_write_kb":4440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":218,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.283014+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:58:39.283014+0000","last_peered":"2024-04-23T01:58:39.283014+0000","last_clean":"2024-04-23T01:58:39.283014+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:58:39.283014+0000","last_undegraded":"2024-04-23T01:58:39.283014+0000","last_fullsized":"2024-04-23T01:58:39.283014+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"92'535","reported_seq":572,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:55.862060+0000","last_change":"2024-04-23T01:57:51.632423+0000","last_active":"2024-04-23T01:58:55.862060+0000","last_peered":"2024-04-23T01:58:55.862060+0000","last_clean":"2024-04-23T01:58:55.862060+0000","last_became_active":"2024-04-23T01:57:51.624354+0000","last_became_peered":"2024-04-23T01:57:51.624354+0000","last_unstale":"2024-04-23T01:58:55.862060+0000","last_undegraded":"2024-04-23T01:58:55.862060+0000","last_fullsized":"2024-04-23T01:58:55.862060+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4390912,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":536,"num_write_kb":4288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":220,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.604585+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:58:39.604585+0000","last_peered":"2024-04-23T01:58:39.604585+0000","last_clean":"2024-04-23T01:58:39.604585+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:58:39.604585+0000","last_undegraded":"2024-04-23T01:58:39.604585+0000","last_fullsized":"2024-04-23T01:58:39.604585+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"92'495","reported_seq":532,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:55.923232+0000","last_change":"2024-04-23T01:57:51.735746+0000","last_active":"2024-04-23T01:58:55.923232+0000","last_peered":"2024-04-23T01:58:55.923232+0000","last_clean":"2024-04-23T01:58:55.923232+0000","last_became_active":"2024-04-23T01:57:51.728785+0000","last_became_peered":"2024-04-23T01:57:51.728785+0000","last_unstale":"2024-04-23T01:58:55.923232+0000","last_undegraded":"2024-04-23T01:58:55.923232+0000","last_fullsized":"2024-04-23T01:58:55.923232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4063232,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":496,"num_write_kb":3968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":218,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.283528+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:58:39.283528+0000","last_peered":"2024-04-23T01:58:39.283528+0000","last_clean":"2024-04-23T01:58:39.283528+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:58:39.283528+0000","last_undegraded":"2024-04-23T01:58:39.283528+0000","last_fullsized":"2024-04-23T01:58:39.283528+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"92'407","reported_seq":438,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:56.370997+0000","last_change":"2024-04-23T01:57:51.656207+0000","last_active":"2024-04-23T01:58:56.370997+0000","last_peered":"2024-04-23T01:58:56.370997+0000","last_clean":"2024-04-23T01:58:56.370997+0000","last_became_active":"2024-04-23T01:57:51.638689+0000","last_became_peered":"2024-04-23T01:57:51.638689+0000","last_unstale":"2024-04-23T01:58:56.370997+0000","last_undegraded":"2024-04-23T01:58:56.370997+0000","last_fullsized":"2024-04-23T01:58:56.370997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342336,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":408,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":218,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.624179+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:58:39.624179+0000","last_peered":"2024-04-23T01:58:39.624179+0000","last_clean":"2024-04-23T01:58:39.624179+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:58:39.624179+0000","last_undegraded":"2024-04-23T01:58:39.624179+0000","last_fullsized":"2024-04-23T01:58:39.624179+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":72523776,"num_objects":1108,"num_object_clones":0,"num_object_copies":3324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8853,"num_write_kb":70824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":108806144,"data_stored":217612288,"data_compressed":11233242,"data_compressed_allocated":108806144,"data_compressed_original":217612288,"omap_allocated":0,"internal_metadata":0},"log_size":8837,"ondisk_log_size":8837,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706492,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":150628,"kb_used_data":21084,"kb_used_omap":0,"kb_used_meta":129536,"kb_avail":94221212,"statfs":{"total":96636764160,"available":96482521088,"internally_reserved":0,"allocated":21590016,"data_stored":42366365,"data_compressed":2171429,"data_compressed_allocated":20783104,"data_compressed_original":42057728,"omap_allocated":0,"internal_metadata":132644864},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706493,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":134416,"kb_used_data":31236,"kb_used_omap":0,"kb_used_meta":103168,"kb_avail":94237424,"statfs":{"total":96636764160,"available":96499122176,"internally_reserved":0,"allocated":31985664,"data_stored":63157661,"data_compressed":3244609,"data_compressed_allocated":31178752,"data_compressed_original":62849024,"omap_allocated":0,"internal_metadata":105644032},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706492,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":147456,"kb_used_data":26232,"kb_used_omap":0,"kb_used_meta":121216,"kb_avail":94224384,"statfs":{"total":96636764160,"available":96485769216,"internally_reserved":0,"allocated":26861568,"data_stored":52441981,"data_compressed":2691932,"data_compressed_allocated":26075136,"data_compressed_original":52150272,"omap_allocated":0,"internal_metadata":124125184},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706492,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":146944,"kb_used_data":29880,"kb_used_omap":0,"kb_used_meta":117056,"kb_avail":94224896,"statfs":{"total":96636764160,"available":96486293504,"internally_reserved":0,"allocated":30597120,"data_stored":60380573,"data_compressed":3101344,"data_compressed_allocated":29790208,"data_compressed_original":60071936,"omap_allocated":0,"internal_metadata":119865344},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":30117888,"data_stored":60235776,"data_compressed":3109461,"data_compressed_allocated":30117888,"data_compressed_original":60235776,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26435584,"data_stored":52871168,"data_compressed":2729156,"data_compressed_allocated":26435584,"data_compressed_original":52871168,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":31277056,"data_stored":62554112,"data_compressed":3229038,"data_compressed_allocated":31277056,"data_compressed_original":62554112,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20975616,"data_stored":41951232,"data_compressed":2165587,"data_compressed_allocated":20975616,"data_compressed_original":41951232,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:59:03.487 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 71 16 9699 9683 1.06538 0.859375 0.134735 0.117197 2024-04-23T01:59:04.303 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:59:04.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:04.415+0000 7f01464ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:04.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:04.422+0000 7f01464ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:04.487 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 72 16 9811 9795 1.06273 0.875 0.171297 0.117487 2024-04-23T01:59:05.487 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 73 16 9942 9926 1.06219 1.02344 0.0295952 0.117583 2024-04-23T01:59:06.487 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 74 16 10059 10043 1.06019 0.914062 0.255275 0.117779 2024-04-23T01:59:06.598 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:59:06.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:06.734+0000 7fbe5bc06640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:06.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:06.740+0000 7fbe5bc06640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:07.487 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 75 16 10184 10168 1.05907 0.976562 0.0378576 0.117913 2024-04-23T01:59:08.403 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:59:08.403 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:59:08.403 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:59:08.403 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:59:08.487 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 76 16 10299 10283 1.05696 0.898438 0.155953 0.118111 2024-04-23T01:59:08.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:08.518+0000 7f205e59d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:08.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:08.529+0000 7f205e59d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:08.553 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:59:08.553 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:59:08.553 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:59:08.553 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:59:08.554 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:59:08.554 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:59:08.554 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:59:08.554 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:59:08.554 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:59:08.554 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:59:08.554 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:59:08.554 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:59:08.557 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:59:08.557 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:59:08.557 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:59:08.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:08.694+0000 7fc8d0ead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:08.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:08.706+0000 7fc8d0ead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:08.734 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:59:08.734 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:59:08.734 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:59:08.734 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:59:08.734 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:59:08.734 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:59:08.734 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:59:08.734 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:59:08.734 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:59:08.735 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:59:08.735 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:59:08.735 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:59:08.737 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:59:08.738 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:59:08.738 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:59:08.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:08.851+0000 7f7ff38ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:08.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:08.862+0000 7f7ff38ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:08.901 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:59:08.901 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:59:08.901 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:59:08.901 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:59:08.902 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:59:08.902 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:59:08.902 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:59:08.902 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:59:08.902 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:59:08.902 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:59:08.902 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:59:08.902 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:59:08.904 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:59:08.904 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:59:08.904 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:59:08.960 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:59:09.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:09.024+0000 7fbd91eed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:09.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:09.036+0000 7fbd91eed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:09.058 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:59:09.059 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:59:09.059 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:59:09.059 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:59:09.059 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:59:09.059 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:59:09.059 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:59:09.060 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:59:09.060 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:59:09.060 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:59:09.060 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:59:09.060 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:59:09.061 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:59:09.061 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:59:09.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:09.089+0000 7f5c65a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:09.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:09.095+0000 7f5c65a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:09.487 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 77 16 10422 10406 1.05571 0.960938 0.165166 0.118266 2024-04-23T01:59:10.487 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 78 16 10555 10539 1.05549 1.03906 0.138865 0.118326 2024-04-23T01:59:11.286 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:59:11.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:11.397+0000 7fca0199d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:11.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:11.402+0000 7fca0199d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:11.488 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 79 16 10666 10650 1.05311 0.867188 0.168886 0.118526 2024-04-23T01:59:12.488 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:2024-04-23T01:59:12.486994+0000 min lat: 0.0138177 max lat: 0.391438 avg lat: 0.118675 2024-04-23T01:59:12.488 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-23T01:59:12.488 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 80 16 10795 10779 1.05254 1.00781 0.0768079 0.118675 2024-04-23T01:59:13.488 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 81 16 10927 10911 1.05228 1.03125 0.179423 0.118684 2024-04-23T01:59:13.565 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:59:13.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:13.695+0000 7f69314ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:13.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:13.701+0000 7f69314ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:14.063 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:59:14.063 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:59:14.063 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:59:14.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:14.201+0000 7fb02bb1d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:14.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:14.214+0000 7fb02bb1d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:14.474 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:59:14.475 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:59:14.488 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 82 16 11059 11043 1.05202 1.03125 0.102149 0.118714 2024-04-23T01:59:14.490 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":509,"stamp":"2024-04-23T01:59:13.714182+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":88039968,"num_objects":1338,"num_object_clones":0,"num_object_copies":4014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":10812,"num_write_kb":87294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10756,"ondisk_log_size":10756,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":627900,"kb_used_data":130524,"kb_used_omap":0,"kb_used_meta":497344,"kb_avail":376859460,"statfs":{"total":386547056640,"available":385904087040,"internally_reserved":0,"allocated":133656576,"data_stored":263590996,"data_compressed":13545543,"data_compressed_allocated":130449408,"data_compressed_original":262373376,"omap_allocated":0,"internal_metadata":509280256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14925824,"num_objects":228,"num_object_clones":0,"num_object_copies":684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1822,"num_write_kb":14576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002324"},"pg_stats":[{"pgid":"10.7","version":"92'692","reported_seq":729,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:11.650753+0000","last_change":"2024-04-23T01:57:51.583793+0000","last_active":"2024-04-23T01:59:11.650753+0000","last_peered":"2024-04-23T01:59:11.650753+0000","last_clean":"2024-04-23T01:59:11.650753+0000","last_became_active":"2024-04-23T01:57:51.574869+0000","last_became_peered":"2024-04-23T01:57:51.574869+0000","last_unstale":"2024-04-23T01:59:11.650753+0000","last_undegraded":"2024-04-23T01:59:11.650753+0000","last_fullsized":"2024-04-23T01:59:11.650753+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5677056,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":693,"num_write_kb":5544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"92'703","reported_seq":734,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:08.582077+0000","last_change":"2024-04-23T01:57:51.679265+0000","last_active":"2024-04-23T01:59:08.582077+0000","last_peered":"2024-04-23T01:59:08.582077+0000","last_clean":"2024-04-23T01:59:08.582077+0000","last_became_active":"2024-04-23T01:57:51.662859+0000","last_became_peered":"2024-04-23T01:57:51.662859+0000","last_unstale":"2024-04-23T01:59:08.582077+0000","last_undegraded":"2024-04-23T01:59:08.582077+0000","last_fullsized":"2024-04-23T01:59:08.582077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767168,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":704,"num_write_kb":5632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"92'620","reported_seq":653,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:11.728552+0000","last_change":"2024-04-23T01:57:51.593118+0000","last_active":"2024-04-23T01:59:11.728552+0000","last_peered":"2024-04-23T01:59:11.728552+0000","last_clean":"2024-04-23T01:59:11.728552+0000","last_became_active":"2024-04-23T01:57:51.580553+0000","last_became_peered":"2024-04-23T01:57:51.580553+0000","last_unstale":"2024-04-23T01:59:11.728552+0000","last_undegraded":"2024-04-23T01:59:11.728552+0000","last_fullsized":"2024-04-23T01:59:11.728552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5087232,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":621,"num_write_kb":4968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"92'599","reported_seq":634,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:10.098170+0000","last_change":"2024-04-23T01:57:51.577069+0000","last_active":"2024-04-23T01:59:10.098170+0000","last_peered":"2024-04-23T01:59:10.098170+0000","last_clean":"2024-04-23T01:59:10.098170+0000","last_became_active":"2024-04-23T01:57:51.568227+0000","last_became_peered":"2024-04-23T01:57:51.568227+0000","last_unstale":"2024-04-23T01:59:10.098170+0000","last_undegraded":"2024-04-23T01:59:10.098170+0000","last_fullsized":"2024-04-23T01:59:10.098170+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"92'815","reported_seq":852,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:11.445405+0000","last_change":"2024-04-23T01:57:51.748592+0000","last_active":"2024-04-23T01:59:11.445405+0000","last_peered":"2024-04-23T01:59:11.445405+0000","last_clean":"2024-04-23T01:59:11.445405+0000","last_became_active":"2024-04-23T01:57:51.742159+0000","last_became_peered":"2024-04-23T01:57:51.742159+0000","last_unstale":"2024-04-23T01:59:11.445405+0000","last_undegraded":"2024-04-23T01:59:11.445405+0000","last_fullsized":"2024-04-23T01:59:11.445405+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6684672,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":816,"num_write_kb":6528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"92'679","reported_seq":714,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:10.636490+0000","last_change":"2024-04-23T01:57:51.630095+0000","last_active":"2024-04-23T01:59:10.636490+0000","last_peered":"2024-04-23T01:59:10.636490+0000","last_clean":"2024-04-23T01:59:10.636490+0000","last_became_active":"2024-04-23T01:57:51.623783+0000","last_became_peered":"2024-04-23T01:57:51.623783+0000","last_unstale":"2024-04-23T01:59:10.636490+0000","last_undegraded":"2024-04-23T01:59:10.636490+0000","last_fullsized":"2024-04-23T01:59:10.636490+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5570560,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":680,"num_write_kb":5440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"92'663","reported_seq":700,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:10.152191+0000","last_change":"2024-04-23T01:57:51.605374+0000","last_active":"2024-04-23T01:59:10.152191+0000","last_peered":"2024-04-23T01:59:10.152191+0000","last_clean":"2024-04-23T01:59:10.152191+0000","last_became_active":"2024-04-23T01:57:51.597053+0000","last_became_peered":"2024-04-23T01:57:51.597053+0000","last_unstale":"2024-04-23T01:59:10.152191+0000","last_undegraded":"2024-04-23T01:59:10.152191+0000","last_fullsized":"2024-04-23T01:59:10.152191+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5439488,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":664,"num_write_kb":5312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"92'799","reported_seq":830,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:11.195127+0000","last_change":"2024-04-23T01:57:51.641687+0000","last_active":"2024-04-23T01:59:11.195127+0000","last_peered":"2024-04-23T01:59:11.195127+0000","last_clean":"2024-04-23T01:59:11.195127+0000","last_became_active":"2024-04-23T01:57:51.623973+0000","last_became_peered":"2024-04-23T01:57:51.623973+0000","last_unstale":"2024-04-23T01:59:11.195127+0000","last_undegraded":"2024-04-23T01:59:11.195127+0000","last_fullsized":"2024-04-23T01:59:11.195127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":799,"log_dups_size":0,"ondisk_log_size":799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6553600,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":800,"num_write_kb":6400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"92'599","reported_seq":630,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:10.455698+0000","last_change":"2024-04-23T01:57:51.669918+0000","last_active":"2024-04-23T01:59:10.455698+0000","last_peered":"2024-04-23T01:59:10.455698+0000","last_clean":"2024-04-23T01:59:10.455698+0000","last_became_active":"2024-04-23T01:57:51.652610+0000","last_became_peered":"2024-04-23T01:57:51.652610+0000","last_unstale":"2024-04-23T01:59:10.455698+0000","last_undegraded":"2024-04-23T01:59:10.455698+0000","last_fullsized":"2024-04-23T01:59:10.455698+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":353,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.603118+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:58:39.603118+0000","last_peered":"2024-04-23T01:58:39.603118+0000","last_clean":"2024-04-23T01:58:39.603118+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:58:39.603118+0000","last_undegraded":"2024-04-23T01:58:39.603118+0000","last_fullsized":"2024-04-23T01:58:39.603118+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":218,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.642652+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:58:39.642652+0000","last_peered":"2024-04-23T01:58:39.642652+0000","last_clean":"2024-04-23T01:58:39.642652+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:58:39.642652+0000","last_undegraded":"2024-04-23T01:58:39.642652+0000","last_fullsized":"2024-04-23T01:58:39.642652+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"92'751","reported_seq":784,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:10.400177+0000","last_change":"2024-04-23T01:57:51.587471+0000","last_active":"2024-04-23T01:59:10.400177+0000","last_peered":"2024-04-23T01:59:10.400177+0000","last_clean":"2024-04-23T01:59:10.400177+0000","last_became_active":"2024-04-23T01:57:51.575352+0000","last_became_peered":"2024-04-23T01:57:51.575352+0000","last_unstale":"2024-04-23T01:59:10.400177+0000","last_undegraded":"2024-04-23T01:59:10.400177+0000","last_fullsized":"2024-04-23T01:59:10.400177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":216,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.610638+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:58:39.610638+0000","last_peered":"2024-04-23T01:58:39.610638+0000","last_clean":"2024-04-23T01:58:39.610638+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:58:39.610638+0000","last_undegraded":"2024-04-23T01:58:39.610638+0000","last_fullsized":"2024-04-23T01:58:39.610638+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"92'680","reported_seq":715,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:08.789227+0000","last_change":"2024-04-23T01:57:51.596134+0000","last_active":"2024-04-23T01:59:08.789227+0000","last_peered":"2024-04-23T01:59:08.789227+0000","last_clean":"2024-04-23T01:59:08.789227+0000","last_became_active":"2024-04-23T01:57:51.589605+0000","last_became_peered":"2024-04-23T01:57:51.589605+0000","last_unstale":"2024-04-23T01:59:08.789227+0000","last_undegraded":"2024-04-23T01:59:08.789227+0000","last_fullsized":"2024-04-23T01:59:08.789227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5578752,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":681,"num_write_kb":5448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":218,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.617202+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:58:39.617202+0000","last_peered":"2024-04-23T01:58:39.617202+0000","last_clean":"2024-04-23T01:58:39.617202+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:58:39.617202+0000","last_undegraded":"2024-04-23T01:58:39.617202+0000","last_fullsized":"2024-04-23T01:58:39.617202+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"92'623","reported_seq":660,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:09.996535+0000","last_change":"2024-04-23T01:57:51.662237+0000","last_active":"2024-04-23T01:59:09.996535+0000","last_peered":"2024-04-23T01:59:09.996535+0000","last_clean":"2024-04-23T01:59:09.996535+0000","last_became_active":"2024-04-23T01:57:51.647583+0000","last_became_peered":"2024-04-23T01:57:51.647583+0000","last_unstale":"2024-04-23T01:59:09.996535+0000","last_undegraded":"2024-04-23T01:59:09.996535+0000","last_fullsized":"2024-04-23T01:59:09.996535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5111808,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":624,"num_write_kb":4992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":216,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.622044+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:58:39.622044+0000","last_peered":"2024-04-23T01:58:39.622044+0000","last_clean":"2024-04-23T01:58:39.622044+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:58:39.622044+0000","last_undegraded":"2024-04-23T01:58:39.622044+0000","last_fullsized":"2024-04-23T01:58:39.622044+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"92'679","reported_seq":716,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:11.541172+0000","last_change":"2024-04-23T01:57:51.681957+0000","last_active":"2024-04-23T01:59:11.541172+0000","last_peered":"2024-04-23T01:59:11.541172+0000","last_clean":"2024-04-23T01:59:11.541172+0000","last_became_active":"2024-04-23T01:57:51.670788+0000","last_became_peered":"2024-04-23T01:57:51.670788+0000","last_unstale":"2024-04-23T01:59:11.541172+0000","last_undegraded":"2024-04-23T01:59:11.541172+0000","last_fullsized":"2024-04-23T01:59:11.541172+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5570560,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":680,"num_write_kb":5440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":218,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.283014+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:58:39.283014+0000","last_peered":"2024-04-23T01:58:39.283014+0000","last_clean":"2024-04-23T01:58:39.283014+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:58:39.283014+0000","last_undegraded":"2024-04-23T01:58:39.283014+0000","last_fullsized":"2024-04-23T01:58:39.283014+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"92'631","reported_seq":668,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:06.375909+0000","last_change":"2024-04-23T01:57:51.632423+0000","last_active":"2024-04-23T01:59:06.375909+0000","last_peered":"2024-04-23T01:59:06.375909+0000","last_clean":"2024-04-23T01:59:06.375909+0000","last_became_active":"2024-04-23T01:57:51.624354+0000","last_became_peered":"2024-04-23T01:57:51.624354+0000","last_unstale":"2024-04-23T01:59:06.375909+0000","last_undegraded":"2024-04-23T01:59:06.375909+0000","last_fullsized":"2024-04-23T01:59:06.375909+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":5056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":220,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.604585+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:58:39.604585+0000","last_peered":"2024-04-23T01:58:39.604585+0000","last_clean":"2024-04-23T01:58:39.604585+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:58:39.604585+0000","last_undegraded":"2024-04-23T01:58:39.604585+0000","last_fullsized":"2024-04-23T01:58:39.604585+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"92'615","reported_seq":652,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:11.628789+0000","last_change":"2024-04-23T01:57:51.735746+0000","last_active":"2024-04-23T01:59:11.628789+0000","last_peered":"2024-04-23T01:59:11.628789+0000","last_clean":"2024-04-23T01:59:11.628789+0000","last_became_active":"2024-04-23T01:57:51.728785+0000","last_became_peered":"2024-04-23T01:57:51.728785+0000","last_unstale":"2024-04-23T01:59:11.628789+0000","last_undegraded":"2024-04-23T01:59:11.628789+0000","last_fullsized":"2024-04-23T01:59:11.628789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5046272,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":616,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":218,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.283528+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:58:39.283528+0000","last_peered":"2024-04-23T01:58:39.283528+0000","last_clean":"2024-04-23T01:58:39.283528+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:58:39.283528+0000","last_undegraded":"2024-04-23T01:58:39.283528+0000","last_fullsized":"2024-04-23T01:58:39.283528+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"92'511","reported_seq":542,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:10.539436+0000","last_change":"2024-04-23T01:57:51.656207+0000","last_active":"2024-04-23T01:59:10.539436+0000","last_peered":"2024-04-23T01:59:10.539436+0000","last_clean":"2024-04-23T01:59:10.539436+0000","last_became_active":"2024-04-23T01:57:51.638689+0000","last_became_peered":"2024-04-23T01:57:51.638689+0000","last_unstale":"2024-04-23T01:59:10.539436+0000","last_undegraded":"2024-04-23T01:59:10.539436+0000","last_fullsized":"2024-04-23T01:59:10.539436+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":512,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":218,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.624179+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:58:39.624179+0000","last_peered":"2024-04-23T01:58:39.624179+0000","last_clean":"2024-04-23T01:58:39.624179+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:58:39.624179+0000","last_undegraded":"2024-04-23T01:58:39.624179+0000","last_fullsized":"2024-04-23T01:58:39.624179+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":87449600,"num_objects":1336,"num_object_clones":0,"num_object_copies":4008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10675,"num_write_kb":85400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131203072,"data_stored":262406144,"data_compressed":13546206,"data_compressed_allocated":131203072,"data_compressed_original":262406144,"omap_allocated":0,"internal_metadata":0},"log_size":10659,"ondisk_log_size":10659,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706507,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":146264,"kb_used_data":25040,"kb_used_omap":0,"kb_used_meta":121216,"kb_avail":94225576,"statfs":{"total":96636764160,"available":96486989824,"internally_reserved":0,"allocated":25640960,"data_stored":50468253,"data_compressed":2589776,"data_compressed_allocated":24834048,"data_compressed_original":50159616,"omap_allocated":0,"internal_metadata":124125184},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706508,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":167572,"kb_used_data":38028,"kb_used_omap":0,"kb_used_meta":129536,"kb_avail":94204268,"statfs":{"total":96636764160,"available":96465170432,"internally_reserved":0,"allocated":38940672,"data_stored":77067677,"data_compressed":3962863,"data_compressed_allocated":38133760,"data_compressed_original":76759040,"omap_allocated":0,"internal_metadata":132644864},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706507,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":156904,"kb_used_data":31520,"kb_used_omap":0,"kb_used_meta":125376,"kb_avail":94214936,"statfs":{"total":96636764160,"available":96476094464,"internally_reserved":0,"allocated":32276480,"data_stored":63271805,"data_compressed":3251138,"data_compressed_allocated":31490048,"data_compressed_original":62980096,"omap_allocated":0,"internal_metadata":128385024},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706507,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":157160,"kb_used_data":35936,"kb_used_omap":0,"kb_used_meta":121216,"kb_avail":94214680,"statfs":{"total":96636764160,"available":96475832320,"internally_reserved":0,"allocated":36798464,"data_stored":72783261,"data_compressed":3741766,"data_compressed_allocated":35991552,"data_compressed_original":72474624,"omap_allocated":0,"internal_metadata":124125184},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36265984,"data_stored":72531968,"data_compressed":3744384,"data_compressed_allocated":36265984,"data_compressed_original":72531968,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":31670272,"data_stored":63340544,"data_compressed":3269750,"data_compressed_allocated":31670272,"data_compressed_original":63340544,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":38178816,"data_stored":76357632,"data_compressed":3941793,"data_compressed_allocated":38178816,"data_compressed_original":76357632,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":25088000,"data_stored":50176000,"data_compressed":2590279,"data_compressed_allocated":25088000,"data_compressed_original":50176000,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:59:15.488 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 83 16 11184 11168 1.05111 0.976562 0.165983 0.118805 2024-04-23T01:59:15.884 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:59:15.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:15.992+0000 7fe685ced640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:16.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:15.997+0000 7fe685ced640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:16.488 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 84 16 11313 11297 1.05059 1.00781 0.179989 0.118877 2024-04-23T01:59:17.488 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 85 16 11426 11410 1.04862 0.882812 0.160392 0.119092 2024-04-23T01:59:18.159 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:59:18.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:18.296+0000 7fc4a779d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:18.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:18.303+0000 7fc4a779d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:18.488 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 86 16 11556 11540 1.04823 1.01562 0.0773712 0.119144 2024-04-23T01:59:19.488 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 87 16 11693 11677 1.04849 1.07031 0.0611803 0.119104 2024-04-23T01:59:19.492 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:59:19.492 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-23T01:59:19.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:19.608+0000 7fa6af06d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:19.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:19.619+0000 7fa6af06d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:20.357 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.0 to 1 (10000) 2024-04-23T01:59:20.373 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-23T01:59:20.438 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:59:20.488 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 88 16 11809 11793 1.04687 0.90625 0.23699 0.119261 2024-04-23T01:59:20.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:20.498+0000 7f148a82d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:20.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:20.511+0000 7f148a82d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:20.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:20.571+0000 7fefa1b9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:20.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:20.579+0000 7fefa1b9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:21.359 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.1 to 1 (10000) 2024-04-23T01:59:21.377 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-23T01:59:21.488 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 89 16 11929 11913 1.04564 0.9375 0.147604 0.119435 2024-04-23T01:59:21.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:21.504+0000 7f4e83b5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:21.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:21.514+0000 7f4e83b5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:22.364 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.2 to 1 (10000) 2024-04-23T01:59:22.379 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-23T01:59:22.488 INFO:tasks.radosbench.radosbench.0.smithi090.stdout: 90 14 12035 12021 1.0434 0.84375 0.0317467 0.119685 2024-04-23T01:59:22.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:22.509+0000 7f4bb766d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:22.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:22.521+0000 7f4bb766d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:22.784 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:59:22.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:22.886+0000 7f7d3659d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:22.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:22.892+0000 7f7d3659d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:23.369 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.3 to 1 (10000) 2024-04-23T01:59:23.382 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-23T01:59:23.382 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:59:23.490 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Total time run: 90.1264 2024-04-23T01:59:23.490 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Total writes made: 12035 2024-04-23T01:59:23.490 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Write size: 8192 2024-04-23T01:59:23.490 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Object size: 65536 2024-04-23T01:59:23.490 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Bandwidth (MB/sec): 1.04324 2024-04-23T01:59:23.490 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Stddev Bandwidth: 0.108329 2024-04-23T01:59:23.490 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Max bandwidth (MB/sec): 1.26562 2024-04-23T01:59:23.490 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Min bandwidth (MB/sec): 0.804688 2024-04-23T01:59:23.490 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Average IOPS: 133 2024-04-23T01:59:23.491 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Stddev IOPS: 13.8662 2024-04-23T01:59:23.491 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Max IOPS: 162 2024-04-23T01:59:23.491 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Min IOPS: 103 2024-04-23T01:59:23.491 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Average Latency(s): 0.119721 2024-04-23T01:59:23.491 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Stddev Latency(s): 0.0600497 2024-04-23T01:59:23.491 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Max latency(s): 0.391438 2024-04-23T01:59:23.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:23.483+0000 7fe1900ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:23.491 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Min latency(s): 0.0138177 2024-04-23T01:59:23.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:23.491+0000 7fe1900ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:23.511 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Cleaning up (deleting benchmark objects) 2024-04-23T01:59:23.727 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:59:23.727 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:59:23.748 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":518,"stamp":"2024-04-23T01:59:23.716219+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":98378272,"num_objects":1496,"num_object_clones":0,"num_object_copies":4488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":12074,"num_write_kb":97390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12023,"ondisk_log_size":12023,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":638916,"kb_used_data":145700,"kb_used_omap":0,"kb_used_meta":493184,"kb_avail":376848444,"statfs":{"total":386547056640,"available":385892806656,"internally_reserved":0,"allocated":149196800,"data_stored":294658305,"data_compressed":15149556,"data_compressed_allocated":145981440,"data_compressed_original":293437440,"omap_allocated":0,"internal_metadata":505020416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4923392,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":601,"num_write_kb":4808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000749"},"pg_stats":[{"pgid":"10.7","version":"93'752","reported_seq":794,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.405576+0000","last_change":"2024-04-23T01:57:51.583793+0000","last_active":"2024-04-23T01:59:21.405576+0000","last_peered":"2024-04-23T01:59:21.405576+0000","last_clean":"2024-04-23T01:59:21.405576+0000","last_became_active":"2024-04-23T01:57:51.574869+0000","last_became_peered":"2024-04-23T01:57:51.574869+0000","last_unstale":"2024-04-23T01:59:21.405576+0000","last_undegraded":"2024-04-23T01:59:21.405576+0000","last_fullsized":"2024-04-23T01:59:21.405576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":752,"log_dups_size":0,"ondisk_log_size":752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"92'791","reported_seq":822,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.582141+0000","last_change":"2024-04-23T01:57:51.679265+0000","last_active":"2024-04-23T01:59:21.582141+0000","last_peered":"2024-04-23T01:59:21.582141+0000","last_clean":"2024-04-23T01:59:21.582141+0000","last_became_active":"2024-04-23T01:57:51.662859+0000","last_became_peered":"2024-04-23T01:57:51.662859+0000","last_unstale":"2024-04-23T01:59:21.582141+0000","last_undegraded":"2024-04-23T01:59:21.582141+0000","last_fullsized":"2024-04-23T01:59:21.582141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6488064,"num_objects":99,"num_object_clones":0,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":792,"num_write_kb":6336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"92'734","reported_seq":767,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.755978+0000","last_change":"2024-04-23T01:57:51.593118+0000","last_active":"2024-04-23T01:59:21.755978+0000","last_peered":"2024-04-23T01:59:21.755978+0000","last_clean":"2024-04-23T01:59:21.755978+0000","last_became_active":"2024-04-23T01:57:51.580553+0000","last_became_peered":"2024-04-23T01:57:51.580553+0000","last_unstale":"2024-04-23T01:59:21.755978+0000","last_undegraded":"2024-04-23T01:59:21.755978+0000","last_fullsized":"2024-04-23T01:59:21.755978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":734,"log_dups_size":0,"ondisk_log_size":734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6021120,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":735,"num_write_kb":5880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"92'655","reported_seq":690,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.434677+0000","last_change":"2024-04-23T01:57:51.577069+0000","last_active":"2024-04-23T01:59:21.434677+0000","last_peered":"2024-04-23T01:59:21.434677+0000","last_clean":"2024-04-23T01:59:21.434677+0000","last_became_active":"2024-04-23T01:57:51.568227+0000","last_became_peered":"2024-04-23T01:57:51.568227+0000","last_unstale":"2024-04-23T01:59:21.434677+0000","last_undegraded":"2024-04-23T01:59:21.434677+0000","last_fullsized":"2024-04-23T01:59:21.434677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"92'872","reported_seq":914,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.414885+0000","last_change":"2024-04-23T01:57:51.748592+0000","last_active":"2024-04-23T01:59:21.414885+0000","last_peered":"2024-04-23T01:59:21.414885+0000","last_clean":"2024-04-23T01:59:21.414885+0000","last_became_active":"2024-04-23T01:57:51.742159+0000","last_became_peered":"2024-04-23T01:57:51.742159+0000","last_unstale":"2024-04-23T01:59:21.414885+0000","last_undegraded":"2024-04-23T01:59:21.414885+0000","last_fullsized":"2024-04-23T01:59:21.414885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":872,"log_dups_size":0,"ondisk_log_size":872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7143424,"num_objects":109,"num_object_clones":0,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":872,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"92'783","reported_seq":818,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:20.800222+0000","last_change":"2024-04-23T01:57:51.630095+0000","last_active":"2024-04-23T01:59:20.800222+0000","last_peered":"2024-04-23T01:59:20.800222+0000","last_clean":"2024-04-23T01:59:20.800222+0000","last_became_active":"2024-04-23T01:57:51.623783+0000","last_became_peered":"2024-04-23T01:57:51.623783+0000","last_unstale":"2024-04-23T01:59:20.800222+0000","last_undegraded":"2024-04-23T01:59:20.800222+0000","last_fullsized":"2024-04-23T01:59:20.800222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6422528,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"93'720","reported_seq":762,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.384450+0000","last_change":"2024-04-23T01:57:51.605374+0000","last_active":"2024-04-23T01:59:21.384450+0000","last_peered":"2024-04-23T01:59:21.384450+0000","last_clean":"2024-04-23T01:59:21.384450+0000","last_became_active":"2024-04-23T01:57:51.597053+0000","last_became_peered":"2024-04-23T01:57:51.597053+0000","last_unstale":"2024-04-23T01:59:21.384450+0000","last_undegraded":"2024-04-23T01:59:21.384450+0000","last_fullsized":"2024-04-23T01:59:21.384450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":720,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"92'855","reported_seq":886,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.256955+0000","last_change":"2024-04-23T01:57:51.641687+0000","last_active":"2024-04-23T01:59:21.256955+0000","last_peered":"2024-04-23T01:59:21.256955+0000","last_clean":"2024-04-23T01:59:21.256955+0000","last_became_active":"2024-04-23T01:57:51.623973+0000","last_became_peered":"2024-04-23T01:57:51.623973+0000","last_unstale":"2024-04-23T01:59:21.256955+0000","last_undegraded":"2024-04-23T01:59:21.256955+0000","last_fullsized":"2024-04-23T01:59:21.256955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7012352,"num_objects":107,"num_object_clones":0,"num_object_copies":321,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":856,"num_write_kb":6848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"92'657","reported_seq":688,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.665875+0000","last_change":"2024-04-23T01:57:51.669918+0000","last_active":"2024-04-23T01:59:21.665875+0000","last_peered":"2024-04-23T01:59:21.665875+0000","last_clean":"2024-04-23T01:59:21.665875+0000","last_became_active":"2024-04-23T01:57:51.652610+0000","last_became_peered":"2024-04-23T01:57:51.652610+0000","last_unstale":"2024-04-23T01:59:21.665875+0000","last_undegraded":"2024-04-23T01:59:21.665875+0000","last_fullsized":"2024-04-23T01:59:21.665875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5390336,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":658,"num_write_kb":5264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":353,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.603118+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:58:39.603118+0000","last_peered":"2024-04-23T01:58:39.603118+0000","last_clean":"2024-04-23T01:58:39.603118+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:58:39.603118+0000","last_undegraded":"2024-04-23T01:58:39.603118+0000","last_fullsized":"2024-04-23T01:58:39.603118+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":218,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.642652+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:58:39.642652+0000","last_peered":"2024-04-23T01:58:39.642652+0000","last_clean":"2024-04-23T01:58:39.642652+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:58:39.642652+0000","last_undegraded":"2024-04-23T01:58:39.642652+0000","last_fullsized":"2024-04-23T01:58:39.642652+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"92'879","reported_seq":912,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.212555+0000","last_change":"2024-04-23T01:57:51.587471+0000","last_active":"2024-04-23T01:59:21.212555+0000","last_peered":"2024-04-23T01:59:21.212555+0000","last_clean":"2024-04-23T01:59:21.212555+0000","last_became_active":"2024-04-23T01:57:51.575352+0000","last_became_peered":"2024-04-23T01:57:51.575352+0000","last_unstale":"2024-04-23T01:59:21.212555+0000","last_undegraded":"2024-04-23T01:59:21.212555+0000","last_fullsized":"2024-04-23T01:59:21.212555+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7208960,"num_objects":110,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":880,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":222,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.394116+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:59:21.394116+0000","last_peered":"2024-04-23T01:59:21.394116+0000","last_clean":"2024-04-23T01:59:21.394116+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:59:21.394116+0000","last_undegraded":"2024-04-23T01:59:21.394116+0000","last_fullsized":"2024-04-23T01:59:21.394116+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"92'767","reported_seq":802,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:19.538419+0000","last_change":"2024-04-23T01:57:51.596134+0000","last_active":"2024-04-23T01:59:19.538419+0000","last_peered":"2024-04-23T01:59:19.538419+0000","last_clean":"2024-04-23T01:59:19.538419+0000","last_became_active":"2024-04-23T01:57:51.589605+0000","last_became_peered":"2024-04-23T01:57:51.589605+0000","last_unstale":"2024-04-23T01:59:19.538419+0000","last_undegraded":"2024-04-23T01:59:19.538419+0000","last_fullsized":"2024-04-23T01:59:19.538419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":768,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":218,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.617202+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:58:39.617202+0000","last_peered":"2024-04-23T01:58:39.617202+0000","last_clean":"2024-04-23T01:58:39.617202+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:58:39.617202+0000","last_undegraded":"2024-04-23T01:58:39.617202+0000","last_fullsized":"2024-04-23T01:58:39.617202+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"93'744","reported_seq":786,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.403816+0000","last_change":"2024-04-23T01:57:51.662237+0000","last_active":"2024-04-23T01:59:21.403816+0000","last_peered":"2024-04-23T01:59:21.403816+0000","last_clean":"2024-04-23T01:59:21.403816+0000","last_became_active":"2024-04-23T01:57:51.647583+0000","last_became_peered":"2024-04-23T01:57:51.647583+0000","last_unstale":"2024-04-23T01:59:21.403816+0000","last_undegraded":"2024-04-23T01:59:21.403816+0000","last_fullsized":"2024-04-23T01:59:21.403816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":744,"log_dups_size":0,"ondisk_log_size":744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6094848,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":744,"num_write_kb":5952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":222,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.409445+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:59:21.409445+0000","last_peered":"2024-04-23T01:59:21.409445+0000","last_clean":"2024-04-23T01:59:21.409445+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:59:21.409445+0000","last_undegraded":"2024-04-23T01:59:21.409445+0000","last_fullsized":"2024-04-23T01:59:21.409445+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"92'736","reported_seq":779,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.393698+0000","last_change":"2024-04-23T01:57:51.681957+0000","last_active":"2024-04-23T01:59:21.393698+0000","last_peered":"2024-04-23T01:59:21.393698+0000","last_clean":"2024-04-23T01:59:21.393698+0000","last_became_active":"2024-04-23T01:57:51.670788+0000","last_became_peered":"2024-04-23T01:57:51.670788+0000","last_unstale":"2024-04-23T01:59:21.393698+0000","last_undegraded":"2024-04-23T01:59:21.393698+0000","last_fullsized":"2024-04-23T01:59:21.393698+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6037504,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":737,"num_write_kb":5896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":218,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.283014+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:58:39.283014+0000","last_peered":"2024-04-23T01:58:39.283014+0000","last_clean":"2024-04-23T01:58:39.283014+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:58:39.283014+0000","last_undegraded":"2024-04-23T01:58:39.283014+0000","last_fullsized":"2024-04-23T01:58:39.283014+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"92'680","reported_seq":722,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.397593+0000","last_change":"2024-04-23T01:57:51.632423+0000","last_active":"2024-04-23T01:59:21.397593+0000","last_peered":"2024-04-23T01:59:21.397593+0000","last_clean":"2024-04-23T01:59:21.397593+0000","last_became_active":"2024-04-23T01:57:51.624354+0000","last_became_peered":"2024-04-23T01:57:51.624354+0000","last_unstale":"2024-04-23T01:59:21.397593+0000","last_undegraded":"2024-04-23T01:59:21.397593+0000","last_fullsized":"2024-04-23T01:59:21.397593+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5570560,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":680,"num_write_kb":5440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":220,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.604585+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:58:39.604585+0000","last_peered":"2024-04-23T01:58:39.604585+0000","last_clean":"2024-04-23T01:58:39.604585+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:58:39.604585+0000","last_undegraded":"2024-04-23T01:58:39.604585+0000","last_fullsized":"2024-04-23T01:58:39.604585+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"93'679","reported_seq":722,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.391053+0000","last_change":"2024-04-23T01:57:51.735746+0000","last_active":"2024-04-23T01:59:21.391053+0000","last_peered":"2024-04-23T01:59:21.391053+0000","last_clean":"2024-04-23T01:59:21.391053+0000","last_became_active":"2024-04-23T01:57:51.728785+0000","last_became_peered":"2024-04-23T01:57:51.728785+0000","last_unstale":"2024-04-23T01:59:21.391053+0000","last_undegraded":"2024-04-23T01:59:21.391053+0000","last_fullsized":"2024-04-23T01:59:21.391053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5570560,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":680,"num_write_kb":5440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":218,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.283528+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:58:39.283528+0000","last_peered":"2024-04-23T01:58:39.283528+0000","last_clean":"2024-04-23T01:58:39.283528+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:58:39.283528+0000","last_undegraded":"2024-04-23T01:58:39.283528+0000","last_fullsized":"2024-04-23T01:58:39.283528+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"92'622","reported_seq":653,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.653773+0000","last_change":"2024-04-23T01:57:51.656207+0000","last_active":"2024-04-23T01:59:21.653773+0000","last_peered":"2024-04-23T01:59:21.653773+0000","last_clean":"2024-04-23T01:59:21.653773+0000","last_became_active":"2024-04-23T01:57:51.638689+0000","last_became_peered":"2024-04-23T01:57:51.638689+0000","last_unstale":"2024-04-23T01:59:21.653773+0000","last_undegraded":"2024-04-23T01:59:21.653773+0000","last_fullsized":"2024-04-23T01:59:21.653773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5103616,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":623,"num_write_kb":4984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":218,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.624179+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:58:39.624179+0000","last_peered":"2024-04-23T01:58:39.624179+0000","last_clean":"2024-04-23T01:58:39.624179+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:58:39.624179+0000","last_undegraded":"2024-04-23T01:58:39.624179+0000","last_fullsized":"2024-04-23T01:58:39.624179+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":97787904,"num_objects":1494,"num_object_clones":0,"num_object_copies":4482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11937,"num_write_kb":95496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":146718720,"data_stored":293437440,"data_compressed":15148527,"data_compressed_allocated":146718720,"data_compressed_original":293437440,"omap_allocated":0,"internal_metadata":0},"log_size":11926,"ondisk_log_size":11926,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":149272,"kb_used_data":28048,"kb_used_omap":0,"kb_used_meta":121216,"kb_avail":94222568,"statfs":{"total":96636764160,"available":96483909632,"internally_reserved":0,"allocated":28721152,"data_stored":56628637,"data_compressed":2907871,"data_compressed_allocated":27914240,"data_compressed_original":56320000,"omap_allocated":0,"internal_metadata":124125184},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":159364,"kb_used_data":42300,"kb_used_omap":0,"kb_used_meta":117056,"kb_avail":94212476,"statfs":{"total":96636764160,"available":96473575424,"internally_reserved":0,"allocated":43315200,"data_stored":85816733,"data_compressed":4414626,"data_compressed_allocated":42508288,"data_compressed_original":85508096,"omap_allocated":0,"internal_metadata":119865344},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":164784,"kb_used_data":35240,"kb_used_omap":0,"kb_used_meta":129536,"kb_avail":94207056,"statfs":{"total":96636764160,"available":96468025344,"internally_reserved":0,"allocated":36085760,"data_stored":70890365,"data_compressed":3644528,"data_compressed_allocated":35299328,"data_compressed_original":70598656,"omap_allocated":0,"internal_metadata":132644864},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":165496,"kb_used_data":40112,"kb_used_omap":0,"kb_used_meta":125376,"kb_avail":94206344,"statfs":{"total":96636764160,"available":96467296256,"internally_reserved":0,"allocated":41074688,"data_stored":81322570,"data_compressed":4182531,"data_compressed_allocated":40259584,"data_compressed_original":81010688,"omap_allocated":0,"internal_metadata":128385024},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40501248,"data_stored":81002496,"data_compressed":4181765,"data_compressed_allocated":40501248,"data_compressed_original":81002496,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":35610624,"data_stored":71221248,"data_compressed":3676676,"data_compressed_allocated":35610624,"data_compressed_original":71221248,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":42569728,"data_stored":85139456,"data_compressed":4395248,"data_compressed_allocated":42569728,"data_compressed_original":85139456,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28037120,"data_stored":56074240,"data_compressed":2894838,"data_compressed_allocated":28037120,"data_compressed_original":56074240,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:59:23.750 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:59:23.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:23.878+0000 7f74c642d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:23.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:23.891+0000 7f74c642d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:24.121 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:59:24.122 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:59:24.137 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":518,"stamp":"2024-04-23T01:59:23.716219+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":98378272,"num_objects":1496,"num_object_clones":0,"num_object_copies":4488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":12074,"num_write_kb":97390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12023,"ondisk_log_size":12023,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":638916,"kb_used_data":145700,"kb_used_omap":0,"kb_used_meta":493184,"kb_avail":376848444,"statfs":{"total":386547056640,"available":385892806656,"internally_reserved":0,"allocated":149196800,"data_stored":294658305,"data_compressed":15149556,"data_compressed_allocated":145981440,"data_compressed_original":293437440,"omap_allocated":0,"internal_metadata":505020416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4923392,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":601,"num_write_kb":4808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000749"},"pg_stats":[{"pgid":"10.7","version":"93'752","reported_seq":794,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.405576+0000","last_change":"2024-04-23T01:57:51.583793+0000","last_active":"2024-04-23T01:59:21.405576+0000","last_peered":"2024-04-23T01:59:21.405576+0000","last_clean":"2024-04-23T01:59:21.405576+0000","last_became_active":"2024-04-23T01:57:51.574869+0000","last_became_peered":"2024-04-23T01:57:51.574869+0000","last_unstale":"2024-04-23T01:59:21.405576+0000","last_undegraded":"2024-04-23T01:59:21.405576+0000","last_fullsized":"2024-04-23T01:59:21.405576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":752,"log_dups_size":0,"ondisk_log_size":752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"92'791","reported_seq":822,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.582141+0000","last_change":"2024-04-23T01:57:51.679265+0000","last_active":"2024-04-23T01:59:21.582141+0000","last_peered":"2024-04-23T01:59:21.582141+0000","last_clean":"2024-04-23T01:59:21.582141+0000","last_became_active":"2024-04-23T01:57:51.662859+0000","last_became_peered":"2024-04-23T01:57:51.662859+0000","last_unstale":"2024-04-23T01:59:21.582141+0000","last_undegraded":"2024-04-23T01:59:21.582141+0000","last_fullsized":"2024-04-23T01:59:21.582141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6488064,"num_objects":99,"num_object_clones":0,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":792,"num_write_kb":6336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"92'734","reported_seq":767,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.755978+0000","last_change":"2024-04-23T01:57:51.593118+0000","last_active":"2024-04-23T01:59:21.755978+0000","last_peered":"2024-04-23T01:59:21.755978+0000","last_clean":"2024-04-23T01:59:21.755978+0000","last_became_active":"2024-04-23T01:57:51.580553+0000","last_became_peered":"2024-04-23T01:57:51.580553+0000","last_unstale":"2024-04-23T01:59:21.755978+0000","last_undegraded":"2024-04-23T01:59:21.755978+0000","last_fullsized":"2024-04-23T01:59:21.755978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":734,"log_dups_size":0,"ondisk_log_size":734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6021120,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":735,"num_write_kb":5880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"92'655","reported_seq":690,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.434677+0000","last_change":"2024-04-23T01:57:51.577069+0000","last_active":"2024-04-23T01:59:21.434677+0000","last_peered":"2024-04-23T01:59:21.434677+0000","last_clean":"2024-04-23T01:59:21.434677+0000","last_became_active":"2024-04-23T01:57:51.568227+0000","last_became_peered":"2024-04-23T01:57:51.568227+0000","last_unstale":"2024-04-23T01:59:21.434677+0000","last_undegraded":"2024-04-23T01:59:21.434677+0000","last_fullsized":"2024-04-23T01:59:21.434677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"92'872","reported_seq":914,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.414885+0000","last_change":"2024-04-23T01:57:51.748592+0000","last_active":"2024-04-23T01:59:21.414885+0000","last_peered":"2024-04-23T01:59:21.414885+0000","last_clean":"2024-04-23T01:59:21.414885+0000","last_became_active":"2024-04-23T01:57:51.742159+0000","last_became_peered":"2024-04-23T01:57:51.742159+0000","last_unstale":"2024-04-23T01:59:21.414885+0000","last_undegraded":"2024-04-23T01:59:21.414885+0000","last_fullsized":"2024-04-23T01:59:21.414885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":872,"log_dups_size":0,"ondisk_log_size":872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7143424,"num_objects":109,"num_object_clones":0,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":872,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"92'783","reported_seq":818,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:20.800222+0000","last_change":"2024-04-23T01:57:51.630095+0000","last_active":"2024-04-23T01:59:20.800222+0000","last_peered":"2024-04-23T01:59:20.800222+0000","last_clean":"2024-04-23T01:59:20.800222+0000","last_became_active":"2024-04-23T01:57:51.623783+0000","last_became_peered":"2024-04-23T01:57:51.623783+0000","last_unstale":"2024-04-23T01:59:20.800222+0000","last_undegraded":"2024-04-23T01:59:20.800222+0000","last_fullsized":"2024-04-23T01:59:20.800222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6422528,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"93'720","reported_seq":762,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.384450+0000","last_change":"2024-04-23T01:57:51.605374+0000","last_active":"2024-04-23T01:59:21.384450+0000","last_peered":"2024-04-23T01:59:21.384450+0000","last_clean":"2024-04-23T01:59:21.384450+0000","last_became_active":"2024-04-23T01:57:51.597053+0000","last_became_peered":"2024-04-23T01:57:51.597053+0000","last_unstale":"2024-04-23T01:59:21.384450+0000","last_undegraded":"2024-04-23T01:59:21.384450+0000","last_fullsized":"2024-04-23T01:59:21.384450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":720,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"92'855","reported_seq":886,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.256955+0000","last_change":"2024-04-23T01:57:51.641687+0000","last_active":"2024-04-23T01:59:21.256955+0000","last_peered":"2024-04-23T01:59:21.256955+0000","last_clean":"2024-04-23T01:59:21.256955+0000","last_became_active":"2024-04-23T01:57:51.623973+0000","last_became_peered":"2024-04-23T01:57:51.623973+0000","last_unstale":"2024-04-23T01:59:21.256955+0000","last_undegraded":"2024-04-23T01:59:21.256955+0000","last_fullsized":"2024-04-23T01:59:21.256955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7012352,"num_objects":107,"num_object_clones":0,"num_object_copies":321,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":856,"num_write_kb":6848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"92'657","reported_seq":688,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.665875+0000","last_change":"2024-04-23T01:57:51.669918+0000","last_active":"2024-04-23T01:59:21.665875+0000","last_peered":"2024-04-23T01:59:21.665875+0000","last_clean":"2024-04-23T01:59:21.665875+0000","last_became_active":"2024-04-23T01:57:51.652610+0000","last_became_peered":"2024-04-23T01:57:51.652610+0000","last_unstale":"2024-04-23T01:59:21.665875+0000","last_undegraded":"2024-04-23T01:59:21.665875+0000","last_fullsized":"2024-04-23T01:59:21.665875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5390336,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":658,"num_write_kb":5264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":353,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.603118+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:58:39.603118+0000","last_peered":"2024-04-23T01:58:39.603118+0000","last_clean":"2024-04-23T01:58:39.603118+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:58:39.603118+0000","last_undegraded":"2024-04-23T01:58:39.603118+0000","last_fullsized":"2024-04-23T01:58:39.603118+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":218,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.642652+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:58:39.642652+0000","last_peered":"2024-04-23T01:58:39.642652+0000","last_clean":"2024-04-23T01:58:39.642652+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:58:39.642652+0000","last_undegraded":"2024-04-23T01:58:39.642652+0000","last_fullsized":"2024-04-23T01:58:39.642652+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"92'879","reported_seq":912,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.212555+0000","last_change":"2024-04-23T01:57:51.587471+0000","last_active":"2024-04-23T01:59:21.212555+0000","last_peered":"2024-04-23T01:59:21.212555+0000","last_clean":"2024-04-23T01:59:21.212555+0000","last_became_active":"2024-04-23T01:57:51.575352+0000","last_became_peered":"2024-04-23T01:57:51.575352+0000","last_unstale":"2024-04-23T01:59:21.212555+0000","last_undegraded":"2024-04-23T01:59:21.212555+0000","last_fullsized":"2024-04-23T01:59:21.212555+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7208960,"num_objects":110,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":880,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":222,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.394116+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:59:21.394116+0000","last_peered":"2024-04-23T01:59:21.394116+0000","last_clean":"2024-04-23T01:59:21.394116+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:59:21.394116+0000","last_undegraded":"2024-04-23T01:59:21.394116+0000","last_fullsized":"2024-04-23T01:59:21.394116+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"92'767","reported_seq":802,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:19.538419+0000","last_change":"2024-04-23T01:57:51.596134+0000","last_active":"2024-04-23T01:59:19.538419+0000","last_peered":"2024-04-23T01:59:19.538419+0000","last_clean":"2024-04-23T01:59:19.538419+0000","last_became_active":"2024-04-23T01:57:51.589605+0000","last_became_peered":"2024-04-23T01:57:51.589605+0000","last_unstale":"2024-04-23T01:59:19.538419+0000","last_undegraded":"2024-04-23T01:59:19.538419+0000","last_fullsized":"2024-04-23T01:59:19.538419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":768,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":218,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.617202+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:58:39.617202+0000","last_peered":"2024-04-23T01:58:39.617202+0000","last_clean":"2024-04-23T01:58:39.617202+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:58:39.617202+0000","last_undegraded":"2024-04-23T01:58:39.617202+0000","last_fullsized":"2024-04-23T01:58:39.617202+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"93'744","reported_seq":786,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.403816+0000","last_change":"2024-04-23T01:57:51.662237+0000","last_active":"2024-04-23T01:59:21.403816+0000","last_peered":"2024-04-23T01:59:21.403816+0000","last_clean":"2024-04-23T01:59:21.403816+0000","last_became_active":"2024-04-23T01:57:51.647583+0000","last_became_peered":"2024-04-23T01:57:51.647583+0000","last_unstale":"2024-04-23T01:59:21.403816+0000","last_undegraded":"2024-04-23T01:59:21.403816+0000","last_fullsized":"2024-04-23T01:59:21.403816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":744,"log_dups_size":0,"ondisk_log_size":744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6094848,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":744,"num_write_kb":5952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":222,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.409445+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:59:21.409445+0000","last_peered":"2024-04-23T01:59:21.409445+0000","last_clean":"2024-04-23T01:59:21.409445+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:59:21.409445+0000","last_undegraded":"2024-04-23T01:59:21.409445+0000","last_fullsized":"2024-04-23T01:59:21.409445+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"92'736","reported_seq":779,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.393698+0000","last_change":"2024-04-23T01:57:51.681957+0000","last_active":"2024-04-23T01:59:21.393698+0000","last_peered":"2024-04-23T01:59:21.393698+0000","last_clean":"2024-04-23T01:59:21.393698+0000","last_became_active":"2024-04-23T01:57:51.670788+0000","last_became_peered":"2024-04-23T01:57:51.670788+0000","last_unstale":"2024-04-23T01:59:21.393698+0000","last_undegraded":"2024-04-23T01:59:21.393698+0000","last_fullsized":"2024-04-23T01:59:21.393698+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6037504,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":737,"num_write_kb":5896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":218,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.283014+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:58:39.283014+0000","last_peered":"2024-04-23T01:58:39.283014+0000","last_clean":"2024-04-23T01:58:39.283014+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:58:39.283014+0000","last_undegraded":"2024-04-23T01:58:39.283014+0000","last_fullsized":"2024-04-23T01:58:39.283014+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"92'680","reported_seq":722,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.397593+0000","last_change":"2024-04-23T01:57:51.632423+0000","last_active":"2024-04-23T01:59:21.397593+0000","last_peered":"2024-04-23T01:59:21.397593+0000","last_clean":"2024-04-23T01:59:21.397593+0000","last_became_active":"2024-04-23T01:57:51.624354+0000","last_became_peered":"2024-04-23T01:57:51.624354+0000","last_unstale":"2024-04-23T01:59:21.397593+0000","last_undegraded":"2024-04-23T01:59:21.397593+0000","last_fullsized":"2024-04-23T01:59:21.397593+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5570560,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":680,"num_write_kb":5440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":220,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.604585+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:58:39.604585+0000","last_peered":"2024-04-23T01:58:39.604585+0000","last_clean":"2024-04-23T01:58:39.604585+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:58:39.604585+0000","last_undegraded":"2024-04-23T01:58:39.604585+0000","last_fullsized":"2024-04-23T01:58:39.604585+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"93'679","reported_seq":722,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.391053+0000","last_change":"2024-04-23T01:57:51.735746+0000","last_active":"2024-04-23T01:59:21.391053+0000","last_peered":"2024-04-23T01:59:21.391053+0000","last_clean":"2024-04-23T01:59:21.391053+0000","last_became_active":"2024-04-23T01:57:51.728785+0000","last_became_peered":"2024-04-23T01:57:51.728785+0000","last_unstale":"2024-04-23T01:59:21.391053+0000","last_undegraded":"2024-04-23T01:59:21.391053+0000","last_fullsized":"2024-04-23T01:59:21.391053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5570560,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":680,"num_write_kb":5440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":218,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.283528+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:58:39.283528+0000","last_peered":"2024-04-23T01:58:39.283528+0000","last_clean":"2024-04-23T01:58:39.283528+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:58:39.283528+0000","last_undegraded":"2024-04-23T01:58:39.283528+0000","last_fullsized":"2024-04-23T01:58:39.283528+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"92'622","reported_seq":653,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:59:21.653773+0000","last_change":"2024-04-23T01:57:51.656207+0000","last_active":"2024-04-23T01:59:21.653773+0000","last_peered":"2024-04-23T01:59:21.653773+0000","last_clean":"2024-04-23T01:59:21.653773+0000","last_became_active":"2024-04-23T01:57:51.638689+0000","last_became_peered":"2024-04-23T01:57:51.638689+0000","last_unstale":"2024-04-23T01:59:21.653773+0000","last_undegraded":"2024-04-23T01:59:21.653773+0000","last_fullsized":"2024-04-23T01:59:21.653773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5103616,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":623,"num_write_kb":4984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":218,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-23T01:58:39.624179+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:58:39.624179+0000","last_peered":"2024-04-23T01:58:39.624179+0000","last_clean":"2024-04-23T01:58:39.624179+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:58:39.624179+0000","last_undegraded":"2024-04-23T01:58:39.624179+0000","last_fullsized":"2024-04-23T01:58:39.624179+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":97787904,"num_objects":1494,"num_object_clones":0,"num_object_copies":4482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11937,"num_write_kb":95496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":146718720,"data_stored":293437440,"data_compressed":15148527,"data_compressed_allocated":146718720,"data_compressed_original":293437440,"omap_allocated":0,"internal_metadata":0},"log_size":11926,"ondisk_log_size":11926,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":149272,"kb_used_data":28048,"kb_used_omap":0,"kb_used_meta":121216,"kb_avail":94222568,"statfs":{"total":96636764160,"available":96483909632,"internally_reserved":0,"allocated":28721152,"data_stored":56628637,"data_compressed":2907871,"data_compressed_allocated":27914240,"data_compressed_original":56320000,"omap_allocated":0,"internal_metadata":124125184},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":159364,"kb_used_data":42300,"kb_used_omap":0,"kb_used_meta":117056,"kb_avail":94212476,"statfs":{"total":96636764160,"available":96473575424,"internally_reserved":0,"allocated":43315200,"data_stored":85816733,"data_compressed":4414626,"data_compressed_allocated":42508288,"data_compressed_original":85508096,"omap_allocated":0,"internal_metadata":119865344},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":164784,"kb_used_data":35240,"kb_used_omap":0,"kb_used_meta":129536,"kb_avail":94207056,"statfs":{"total":96636764160,"available":96468025344,"internally_reserved":0,"allocated":36085760,"data_stored":70890365,"data_compressed":3644528,"data_compressed_allocated":35299328,"data_compressed_original":70598656,"omap_allocated":0,"internal_metadata":132644864},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":165496,"kb_used_data":40112,"kb_used_omap":0,"kb_used_meta":125376,"kb_avail":94206344,"statfs":{"total":96636764160,"available":96467296256,"internally_reserved":0,"allocated":41074688,"data_stored":81322570,"data_compressed":4182531,"data_compressed_allocated":40259584,"data_compressed_original":81010688,"omap_allocated":0,"internal_metadata":128385024},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40501248,"data_stored":81002496,"data_compressed":4181765,"data_compressed_allocated":40501248,"data_compressed_original":81002496,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":35610624,"data_stored":71221248,"data_compressed":3676676,"data_compressed_allocated":35610624,"data_compressed_original":71221248,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":42569728,"data_stored":85139456,"data_compressed":4395248,"data_compressed_allocated":42569728,"data_compressed_original":85139456,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28037120,"data_stored":56074240,"data_compressed":2894838,"data_compressed_allocated":28037120,"data_compressed_original":56074240,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:59:24.139 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-23T01:59:24.139 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:59:24.139 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:59:24.139 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:59:24.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:24.263+0000 7f8fc7b9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:24.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:24.275+0000 7f8fc7b9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:24.311 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:59:24.311 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:59:24.311 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:59:24.311 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:59:24.311 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:59:24.311 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:59:24.311 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:59:24.311 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:59:24.311 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:59:24.311 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:59:24.312 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:59:24.312 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:59:24.314 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:59:24.314 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:59:24.314 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:59:24.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:24.436+0000 7f0912ead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:24.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:24.447+0000 7f0912ead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:24.605 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:59:24.605 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:59:24.605 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:59:24.606 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:59:24.606 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:59:24.606 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:59:24.606 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:59:24.606 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:59:24.606 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:59:24.606 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:59:24.606 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:59:24.606 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:59:24.609 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:59:24.609 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:59:24.609 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:59:24.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:24.734+0000 7f11fba64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:24.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:24.746+0000 7f11fba64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:24.763 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:59:24.763 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:59:24.764 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:59:24.764 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:59:24.764 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:59:24.764 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:59:24.764 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:59:24.764 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:59:24.764 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:59:24.764 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:59:24.764 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:59:24.764 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:59:24.766 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:59:24.767 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:59:24.767 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:59:24.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:24.889+0000 7f288626d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:24.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:24.900+0000 7f288626d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:24.949 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:59:24.949 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:59:24.949 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:59:24.949 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:59:24.949 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:59:24.949 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:59:24.949 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:59:24.949 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:59:24.949 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:59:24.949 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:59:24.949 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:59:24.949 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:59:24.951 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:59:24.952 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:59:24.988 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:59:25.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:25.111+0000 7fa7f746d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:25.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:25.119+0000 7fa7f746d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:27.499 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:59:27.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:27.619+0000 7fbc9875d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:27.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:27.629+0000 7fbc9875d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:29.902 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:59:29.953 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:59:29.953 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:59:29.953 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-23T01:59:29.953 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-23T01:59:30.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:30.027+0000 7f09eacad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:30.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:30.034+0000 7f09eacad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:30.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:30.088+0000 7f1f4cc6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:30.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:30.101+0000 7f1f4cc6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:30.255 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:59:30.255 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:59:30.256 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:59:30.256 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:59:30.256 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:59:30.256 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:59:30.256 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:59:30.256 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:59:30.256 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:59:30.256 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:59:30.256 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:59:30.256 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:59:30.259 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:59:30.260 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:59:30.260 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-23T01:59:30.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:30.385+0000 7f16dea2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:30.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:30.400+0000 7f16dea2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:30.447 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:59:30.447 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:59:30.447 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:59:30.447 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:59:30.448 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:59:30.448 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:59:30.448 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:59:30.448 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:59:30.448 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:59:30.448 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:59:30.448 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:59:30.448 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:59:30.451 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:59:30.451 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:59:30.451 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-23T01:59:30.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:30.571+0000 7fe897a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:30.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:30.582+0000 7fe897a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:30.607 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:59:30.607 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:59:30.607 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:59:30.607 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:59:30.607 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:59:30.607 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:59:30.607 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:59:30.607 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:59:30.608 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:59:30.608 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:59:30.608 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:59:30.608 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:59:30.611 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:59:30.611 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:59:30.611 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-23T01:59:30.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:30.779+0000 7f21da59d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:30.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:30.792+0000 7f21da59d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:30.822 INFO:teuthology.orchestra.run.smithi090.stderr:no valid command found; 10 closest matches: 2024-04-23T01:59:30.822 INFO:teuthology.orchestra.run.smithi090.stderr:config get 2024-04-23T01:59:30.822 INFO:teuthology.orchestra.run.smithi090.stderr:config help 2024-04-23T01:59:30.822 INFO:teuthology.orchestra.run.smithi090.stderr:config set ... 2024-04-23T01:59:30.822 INFO:teuthology.orchestra.run.smithi090.stderr:config show 2024-04-23T01:59:30.822 INFO:teuthology.orchestra.run.smithi090.stderr:deep_scrub 2024-04-23T01:59:30.823 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_ops 2024-04-23T01:59:30.823 INFO:teuthology.orchestra.run.smithi090.stderr:dump_historic_slow_ops 2024-04-23T01:59:30.823 INFO:teuthology.orchestra.run.smithi090.stderr:dump_metrics [] 2024-04-23T01:59:30.823 INFO:teuthology.orchestra.run.smithi090.stderr:dump_ops_in_flight 2024-04-23T01:59:30.823 INFO:teuthology.orchestra.run.smithi090.stderr:dump_pgstate_history 2024-04-23T01:59:30.823 INFO:teuthology.orchestra.run.smithi090.stderr:Error EINVAL: invalid command 2024-04-23T01:59:30.826 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T01:59:30.827 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-23T01:59:32.571 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:59:32.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:32.686+0000 7ff5242ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:32.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:32.694+0000 7ff5242ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:33.696 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Removed 1505 objects 2024-04-23T01:59:33.696 INFO:tasks.radosbench.radosbench.0.smithi090.stdout:Clean up completed and total clean up time :10.1853 2024-04-23T01:59:35.021 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-23T01:59:35.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:35.129+0000 7f870b79d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:35.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:35.134+0000 7f870b79d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:35.828 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-04-23T01:59:35.828 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-04-23T01:59:35.828 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:59:35.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:35.930+0000 7f2c89aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:35.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:35.938+0000 7f2c89aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:36.127 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:59:36.127 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:59:36.139 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":524,"stamp":"2024-04-23T01:59:35.718828+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19817016,"num_objects":297,"num_object_clones":0,"num_object_copies":891,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":13384,"num_write_kb":98175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13328,"ondisk_log_size":13328,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":549828,"kb_used_data":39940,"kb_used_omap":0,"kb_used_meta":509824,"kb_avail":376937532,"statfs":{"total":386547056640,"available":385984032768,"internally_reserved":0,"allocated":40898560,"data_stored":77840236,"data_compressed":3954807,"data_compressed_allocated":37548032,"data_compressed_original":76570624,"omap_allocated":0,"internal_metadata":522059776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-78561256,"num_objects":-1199,"num_object_clones":0,"num_object_copies":-3597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1310,"num_write_kb":785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002609"},"pg_stats":[{"pgid":"10.7","version":"96'830","reported_seq":877,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-23T01:59:31.636409+0000","last_change":"2024-04-23T01:57:51.583793+0000","last_active":"2024-04-23T01:59:31.636409+0000","last_peered":"2024-04-23T01:59:31.636409+0000","last_clean":"2024-04-23T01:59:31.636409+0000","last_became_active":"2024-04-23T01:57:51.574869+0000","last_became_peered":"2024-04-23T01:57:51.574869+0000","last_unstale":"2024-04-23T01:59:31.636409+0000","last_undegraded":"2024-04-23T01:59:31.636409+0000","last_fullsized":"2024-04-23T01:59:31.636409+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":830,"log_dups_size":0,"ondisk_log_size":830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":831,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"96'884","reported_seq":921,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-23T01:59:31.360173+0000","last_change":"2024-04-23T01:57:51.679265+0000","last_active":"2024-04-23T01:59:31.360173+0000","last_peered":"2024-04-23T01:59:31.360173+0000","last_clean":"2024-04-23T01:59:31.360173+0000","last_became_active":"2024-04-23T01:57:51.662859+0000","last_became_peered":"2024-04-23T01:57:51.662859+0000","last_unstale":"2024-04-23T01:59:31.360173+0000","last_undegraded":"2024-04-23T01:59:31.360173+0000","last_fullsized":"2024-04-23T01:59:31.360173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1572864,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":885,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"96'803","reported_seq":844,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-23T01:59:31.758490+0000","last_change":"2024-04-23T01:57:51.593118+0000","last_active":"2024-04-23T01:59:31.758490+0000","last_peered":"2024-04-23T01:59:31.758490+0000","last_clean":"2024-04-23T01:59:31.758490+0000","last_became_active":"2024-04-23T01:57:51.580553+0000","last_became_peered":"2024-04-23T01:57:51.580553+0000","last_unstale":"2024-04-23T01:59:31.758490+0000","last_undegraded":"2024-04-23T01:59:31.758490+0000","last_fullsized":"2024-04-23T01:59:31.758490+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1572864,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":804,"num_write_kb":5888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"96'733","reported_seq":776,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-23T01:59:31.446452+0000","last_change":"2024-04-23T01:57:51.577069+0000","last_active":"2024-04-23T01:59:31.446452+0000","last_peered":"2024-04-23T01:59:31.446452+0000","last_clean":"2024-04-23T01:59:31.446452+0000","last_became_active":"2024-04-23T01:57:51.568227+0000","last_became_peered":"2024-04-23T01:57:51.568227+0000","last_unstale":"2024-04-23T01:59:31.446452+0000","last_undegraded":"2024-04-23T01:59:31.446452+0000","last_fullsized":"2024-04-23T01:59:31.446452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":734,"num_write_kb":5312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"96'960","reported_seq":1007,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-23T01:59:31.500766+0000","last_change":"2024-04-23T01:57:51.748592+0000","last_active":"2024-04-23T01:59:31.500766+0000","last_peered":"2024-04-23T01:59:31.500766+0000","last_clean":"2024-04-23T01:59:31.500766+0000","last_became_active":"2024-04-23T01:57:51.742159+0000","last_became_peered":"2024-04-23T01:57:51.742159+0000","last_unstale":"2024-04-23T01:59:31.500766+0000","last_undegraded":"2024-04-23T01:59:31.500766+0000","last_fullsized":"2024-04-23T01:59:31.500766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":960,"log_dups_size":0,"ondisk_log_size":960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310720,"num_objects":20,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":961,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"96'871","reported_seq":914,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-23T01:59:31.340004+0000","last_change":"2024-04-23T01:57:51.630095+0000","last_active":"2024-04-23T01:59:31.340004+0000","last_peered":"2024-04-23T01:59:31.340004+0000","last_clean":"2024-04-23T01:59:31.340004+0000","last_became_active":"2024-04-23T01:57:51.623783+0000","last_became_peered":"2024-04-23T01:57:51.623783+0000","last_unstale":"2024-04-23T01:59:31.340004+0000","last_undegraded":"2024-04-23T01:59:31.340004+0000","last_fullsized":"2024-04-23T01:59:31.340004+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":872,"num_write_kb":6336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"96'796","reported_seq":843,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-23T01:59:31.560001+0000","last_change":"2024-04-23T01:57:51.605374+0000","last_active":"2024-04-23T01:59:31.560001+0000","last_peered":"2024-04-23T01:59:31.560001+0000","last_clean":"2024-04-23T01:59:31.560001+0000","last_became_active":"2024-04-23T01:57:51.597053+0000","last_became_peered":"2024-04-23T01:57:51.597053+0000","last_unstale":"2024-04-23T01:59:31.560001+0000","last_undegraded":"2024-04-23T01:59:31.560001+0000","last_fullsized":"2024-04-23T01:59:31.560001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":796,"log_dups_size":0,"ondisk_log_size":796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":797,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"96'953","reported_seq":990,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-23T01:59:31.427654+0000","last_change":"2024-04-23T01:57:51.641687+0000","last_active":"2024-04-23T01:59:31.427654+0000","last_peered":"2024-04-23T01:59:31.427654+0000","last_clean":"2024-04-23T01:59:31.427654+0000","last_became_active":"2024-04-23T01:57:51.623973+0000","last_became_peered":"2024-04-23T01:57:51.623973+0000","last_unstale":"2024-04-23T01:59:31.427654+0000","last_undegraded":"2024-04-23T01:59:31.427654+0000","last_fullsized":"2024-04-23T01:59:31.427654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":954,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"96'737","reported_seq":774,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-23T01:59:31.248563+0000","last_change":"2024-04-23T01:57:51.669918+0000","last_active":"2024-04-23T01:59:31.248563+0000","last_peered":"2024-04-23T01:59:31.248563+0000","last_clean":"2024-04-23T01:59:31.248563+0000","last_became_active":"2024-04-23T01:57:51.652610+0000","last_became_peered":"2024-04-23T01:57:51.652610+0000","last_unstale":"2024-04-23T01:59:31.248563+0000","last_undegraded":"2024-04-23T01:59:31.248563+0000","last_fullsized":"2024-04-23T01:59:31.248563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":737,"log_dups_size":0,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":738,"num_write_kb":5376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":361,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-23T01:59:23.393058+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:59:23.393058+0000","last_peered":"2024-04-23T01:59:23.393058+0000","last_clean":"2024-04-23T01:59:23.393058+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:59:23.393058+0000","last_undegraded":"2024-04-23T01:59:23.393058+0000","last_fullsized":"2024-04-23T01:59:23.393058+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":224,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-23T01:59:23.885723+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:59:23.885723+0000","last_peered":"2024-04-23T01:59:23.885723+0000","last_clean":"2024-04-23T01:59:23.885723+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:59:23.885723+0000","last_undegraded":"2024-04-23T01:59:23.885723+0000","last_fullsized":"2024-04-23T01:59:23.885723+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"96'975","reported_seq":1016,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-23T01:59:31.500815+0000","last_change":"2024-04-23T01:57:51.587471+0000","last_active":"2024-04-23T01:59:31.500815+0000","last_peered":"2024-04-23T01:59:31.500815+0000","last_clean":"2024-04-23T01:59:31.500815+0000","last_became_active":"2024-04-23T01:57:51.575352+0000","last_became_peered":"2024-04-23T01:57:51.575352+0000","last_unstale":"2024-04-23T01:59:31.500815+0000","last_undegraded":"2024-04-23T01:59:31.500815+0000","last_fullsized":"2024-04-23T01:59:31.500815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":976,"num_write_kb":7104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":226,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-23T01:59:23.792725+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:59:23.792725+0000","last_peered":"2024-04-23T01:59:23.792725+0000","last_clean":"2024-04-23T01:59:23.792725+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:59:23.792725+0000","last_undegraded":"2024-04-23T01:59:23.792725+0000","last_fullsized":"2024-04-23T01:59:23.792725+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"96'847","reported_seq":890,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-23T01:59:31.544789+0000","last_change":"2024-04-23T01:57:51.596134+0000","last_active":"2024-04-23T01:59:31.544789+0000","last_peered":"2024-04-23T01:59:31.544789+0000","last_clean":"2024-04-23T01:59:31.544789+0000","last_became_active":"2024-04-23T01:57:51.589605+0000","last_became_peered":"2024-04-23T01:57:51.589605+0000","last_unstale":"2024-04-23T01:59:31.544789+0000","last_undegraded":"2024-04-23T01:59:31.544789+0000","last_fullsized":"2024-04-23T01:59:31.544789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":848,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":224,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-23T01:59:23.858499+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:59:23.858499+0000","last_peered":"2024-04-23T01:59:23.858499+0000","last_clean":"2024-04-23T01:59:23.858499+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:59:23.858499+0000","last_undegraded":"2024-04-23T01:59:23.858499+0000","last_fullsized":"2024-04-23T01:59:23.858499+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"96'822","reported_seq":869,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-23T01:59:31.539609+0000","last_change":"2024-04-23T01:57:51.662237+0000","last_active":"2024-04-23T01:59:31.539609+0000","last_peered":"2024-04-23T01:59:31.539609+0000","last_clean":"2024-04-23T01:59:31.539609+0000","last_became_active":"2024-04-23T01:57:51.647583+0000","last_became_peered":"2024-04-23T01:57:51.647583+0000","last_unstale":"2024-04-23T01:59:31.539609+0000","last_undegraded":"2024-04-23T01:59:31.539609+0000","last_fullsized":"2024-04-23T01:59:31.539609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":822,"log_dups_size":0,"ondisk_log_size":822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":823,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":226,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-23T01:59:23.770044+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:59:23.770044+0000","last_peered":"2024-04-23T01:59:23.770044+0000","last_clean":"2024-04-23T01:59:23.770044+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:59:23.770044+0000","last_undegraded":"2024-04-23T01:59:23.770044+0000","last_fullsized":"2024-04-23T01:59:23.770044+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"96'828","reported_seq":875,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-23T01:59:31.612124+0000","last_change":"2024-04-23T01:57:51.681957+0000","last_active":"2024-04-23T01:59:31.612124+0000","last_peered":"2024-04-23T01:59:31.612124+0000","last_clean":"2024-04-23T01:59:31.612124+0000","last_became_active":"2024-04-23T01:57:51.670788+0000","last_became_peered":"2024-04-23T01:57:51.670788+0000","last_unstale":"2024-04-23T01:59:31.612124+0000","last_undegraded":"2024-04-23T01:59:31.612124+0000","last_fullsized":"2024-04-23T01:59:31.612124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":828,"log_dups_size":0,"ondisk_log_size":828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":829,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":226,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-23T01:59:23.800063+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:59:23.800063+0000","last_peered":"2024-04-23T01:59:23.800063+0000","last_clean":"2024-04-23T01:59:23.800063+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:59:23.800063+0000","last_undegraded":"2024-04-23T01:59:23.800063+0000","last_fullsized":"2024-04-23T01:59:23.800063+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"96'755","reported_seq":803,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-23T01:59:31.669012+0000","last_change":"2024-04-23T01:57:51.632423+0000","last_active":"2024-04-23T01:59:31.669012+0000","last_peered":"2024-04-23T01:59:31.669012+0000","last_clean":"2024-04-23T01:59:31.669012+0000","last_became_active":"2024-04-23T01:57:51.624354+0000","last_became_peered":"2024-04-23T01:57:51.624354+0000","last_unstale":"2024-04-23T01:59:31.669012+0000","last_undegraded":"2024-04-23T01:59:31.669012+0000","last_fullsized":"2024-04-23T01:59:31.669012+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":755,"log_dups_size":0,"ondisk_log_size":755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655384,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":756,"num_write_kb":5441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":228,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-23T01:59:23.405109+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:59:23.405109+0000","last_peered":"2024-04-23T01:59:23.405109+0000","last_clean":"2024-04-23T01:59:23.405109+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:59:23.405109+0000","last_undegraded":"2024-04-23T01:59:23.405109+0000","last_fullsized":"2024-04-23T01:59:23.405109+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"96'750","reported_seq":797,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-23T01:59:31.581084+0000","last_change":"2024-04-23T01:57:51.735746+0000","last_active":"2024-04-23T01:59:31.581084+0000","last_peered":"2024-04-23T01:59:31.581084+0000","last_clean":"2024-04-23T01:59:31.581084+0000","last_became_active":"2024-04-23T01:57:51.728785+0000","last_became_peered":"2024-04-23T01:57:51.728785+0000","last_unstale":"2024-04-23T01:59:31.581084+0000","last_undegraded":"2024-04-23T01:59:31.581084+0000","last_fullsized":"2024-04-23T01:59:31.581084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1138688,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":751,"num_write_kb":5464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":226,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-23T01:59:23.789281+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:59:23.789281+0000","last_peered":"2024-04-23T01:59:23.789281+0000","last_clean":"2024-04-23T01:59:23.789281+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:59:23.789281+0000","last_undegraded":"2024-04-23T01:59:23.789281+0000","last_fullsized":"2024-04-23T01:59:23.789281+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"96'687","reported_seq":724,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-23T01:59:31.392603+0000","last_change":"2024-04-23T01:57:51.656207+0000","last_active":"2024-04-23T01:59:31.392603+0000","last_peered":"2024-04-23T01:59:31.392603+0000","last_clean":"2024-04-23T01:59:31.392603+0000","last_became_active":"2024-04-23T01:57:51.638689+0000","last_became_peered":"2024-04-23T01:57:51.638689+0000","last_unstale":"2024-04-23T01:59:31.392603+0000","last_undegraded":"2024-04-23T01:59:31.392603+0000","last_fullsized":"2024-04-23T01:59:31.392603+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:57:50.388836+0000","last_clean_scrub_stamp":"2024-04-23T01:57:50.388836+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":688,"num_write_kb":5056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":224,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-23T01:59:23.880221+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:59:23.880221+0000","last_peered":"2024-04-23T01:59:23.880221+0000","last_clean":"2024-04-23T01:59:23.880221+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:59:23.880221+0000","last_undegraded":"2024-04-23T01:59:23.880221+0000","last_fullsized":"2024-04-23T01:59:23.880221+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":19226648,"num_objects":295,"num_object_clones":0,"num_object_copies":885,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13247,"num_write_kb":96281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28721152,"data_stored":57417800,"data_compressed":2964804,"data_compressed_allocated":28708864,"data_compressed_original":57417728,"omap_allocated":0,"internal_metadata":0},"log_size":13231,"ondisk_log_size":13231,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706527,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":133700,"kb_used_data":8316,"kb_used_omap":0,"kb_used_meta":125376,"kb_avail":94238140,"statfs":{"total":96636764160,"available":96499855360,"internally_reserved":0,"allocated":8515584,"data_stored":16164945,"data_compressed":818424,"data_compressed_allocated":7675904,"data_compressed_original":15843328,"omap_allocated":0,"internal_metadata":128385024},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706528,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":130804,"kb_used_data":9548,"kb_used_omap":0,"kb_used_meta":121216,"kb_avail":94241036,"statfs":{"total":96636764160,"available":96502820864,"internally_reserved":0,"allocated":9777152,"data_stored":18679913,"data_compressed":948285,"data_compressed_allocated":8933376,"data_compressed_original":18358272,"omap_allocated":0,"internal_metadata":124125184},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706527,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":144376,"kb_used_data":10672,"kb_used_omap":0,"kb_used_meta":133696,"kb_avail":94227464,"statfs":{"total":96636764160,"available":96488923136,"internally_reserved":0,"allocated":10928128,"data_stored":20514377,"data_compressed":1043541,"data_compressed_allocated":10104832,"data_compressed_original":20209664,"omap_allocated":0,"internal_metadata":136904704},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706527,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":140948,"kb_used_data":11404,"kb_used_omap":0,"kb_used_meta":129536,"kb_avail":94230892,"statfs":{"total":96636764160,"available":96492433408,"internally_reserved":0,"allocated":11677696,"data_stored":22481001,"data_compressed":1144557,"data_compressed_allocated":10833920,"data_compressed_original":22159360,"omap_allocated":0,"internal_metadata":132644864},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7847936,"data_stored":15687704,"data_compressed":810044,"data_compressed_allocated":7843840,"data_compressed_original":15687680,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7225344,"data_stored":14442520,"data_compressed":745749,"data_compressed_allocated":7221248,"data_compressed_original":14442496,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8175616,"data_stored":16343064,"data_compressed":843884,"data_compressed_allocated":8171520,"data_compressed_original":16343040,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5472256,"data_stored":10944512,"data_compressed":565127,"data_compressed_allocated":5472256,"data_compressed_original":10944512,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:59:36.147 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_7 2024-04-23T01:59:36.148 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_7 unique_pool_7 --yes-i-really-really-mean-it 2024-04-23T01:59:36.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:36.248+0000 7f7393e24640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:36.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:36.260+0000 7f7393e24640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:37.236 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-23T01:59:37.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:37.338+0000 7f2b5035d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:37.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:37.343+0000 7f2b5035d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:37.395 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'unique_pool_7' removed 2024-04-23T01:59:37.408 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-04-23T01:59:37.420 INFO:tasks.thrashosds:joining thrashosds 2024-04-23T01:59:41.141 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-04-23T01:59:41.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:41.242+0000 7f749f02d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:41.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:41.247+0000 7f749f02d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:41.264 INFO:teuthology.orchestra.run.smithi090.stdout:{} 2024-04-23T01:59:41.264 INFO:teuthology.orchestra.run.smithi090.stderr:mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-23T01:59:41.274 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-23T01:59:41.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:41.377+0000 7f7253c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:41.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:41.386+0000 7f7253c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:42.407 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.0 to 1 (10000) 2024-04-23T01:59:42.428 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-04-23T01:59:42.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:42.530+0000 7f8598d5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:42.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:42.539+0000 7f8598d5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:43.413 INFO:teuthology.orchestra.run.smithi090.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-04-23T01:59:43.430 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-23T01:59:43.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:43.532+0000 7fbb0a206640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:43.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:43.542+0000 7fbb0a206640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:44.418 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.1 to 1 (10000) 2024-04-23T01:59:44.434 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-04-23T01:59:44.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:44.539+0000 7f1b3ecad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:44.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:44.549+0000 7f1b3ecad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:45.422 INFO:teuthology.orchestra.run.smithi090.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-04-23T01:59:45.439 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-23T01:59:45.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:45.543+0000 7f4e710ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:45.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:45.552+0000 7f4e710ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:46.426 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.2 to 1 (10000) 2024-04-23T01:59:46.442 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-04-23T01:59:46.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:46.545+0000 7fa46db9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:46.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:46.555+0000 7fa46db9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:47.430 INFO:teuthology.orchestra.run.smithi090.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-04-23T01:59:47.446 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-23T01:59:47.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:47.555+0000 7f986d0ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:47.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:47.565+0000 7f986d0ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:48.436 INFO:teuthology.orchestra.run.smithi090.stderr:reweighted osd.3 to 1 (10000) 2024-04-23T01:59:48.452 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-04-23T01:59:48.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:48.555+0000 7f0bd5e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:48.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:48.565+0000 7f0bd5e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:49.437 INFO:teuthology.orchestra.run.smithi090.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-04-23T01:59:49.452 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-04-23T01:59:49.453 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-23T01:59:49.453 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T01:59:49.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:49.554+0000 7fa7d65ff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:49.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:49.561+0000 7fa7d539d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:49.752 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:59:49.752 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":106,"fsid":"0a731bfd-0fc4-4c1b-8e4b-f6dfbc059bcc","created":"2024-04-23T01:44:59.145590+0000","modified":"2024-04-23T01:59:49.435223+0000","last_up_change":"2024-04-23T01:45:06.551915+0000","last_in_change":"2024-04-23T01:45:01.284163+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T01:45:07.564959+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-23T01:45:09.460600+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e81b79-2e2e-4e90-ac13-13a26b240245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6801","nonce":4128231813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":4128231813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6807","nonce":4128231813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":4128231813}]},"public_addr":"172.21.15.90:6801/4128231813","cluster_addr":"172.21.15.90:6802/4128231813","heartbeat_back_addr":"172.21.15.90:6807/4128231813","heartbeat_front_addr":"172.21.15.90:6806/4128231813","state":["exists","up"]},{"osd":1,"uuid":"e81e8b79-a326-41e5-9f97-1da36297e891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6803","nonce":55615551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":55615551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":55615551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":55615551}]},"public_addr":"172.21.15.90:6803/55615551","cluster_addr":"172.21.15.90:6804/55615551","heartbeat_back_addr":"172.21.15.90:6805/55615551","heartbeat_front_addr":"172.21.15.90:6800/55615551","state":["exists","up"]},{"osd":2,"uuid":"a798cb89-96c0-4222-b32e-d08a97970e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":474973433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":474973433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6811","nonce":474973433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":474973433}]},"public_addr":"172.21.15.113:6807/474973433","cluster_addr":"172.21.15.113:6806/474973433","heartbeat_back_addr":"172.21.15.113:6811/474973433","heartbeat_front_addr":"172.21.15.113:6810/474973433","state":["exists","up"]},{"osd":3,"uuid":"446df045-a76b-4937-ae3f-dc84a4e1ca87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6805","nonce":2075119886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":2075119886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":2075119886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6803","nonce":2075119886}]},"public_addr":"172.21.15.113:6805/2075119886","cluster_addr":"172.21.15.113:6804/2075119886","heartbeat_back_addr":"172.21.15.113:6808/2075119886","heartbeat_front_addr":"172.21.15.113:6803/2075119886","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T01:59:49.765 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-23T01:59:49.765 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T01:59:49.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:49.867+0000 7fc26499d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:49.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:49.874+0000 7fc26499d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:50.065 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:59:50.065 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":106,"fsid":"0a731bfd-0fc4-4c1b-8e4b-f6dfbc059bcc","created":"2024-04-23T01:44:59.145590+0000","modified":"2024-04-23T01:59:49.435223+0000","last_up_change":"2024-04-23T01:45:06.551915+0000","last_in_change":"2024-04-23T01:45:01.284163+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T01:45:07.564959+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-23T01:45:09.460600+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e81b79-2e2e-4e90-ac13-13a26b240245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6801","nonce":4128231813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":4128231813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6807","nonce":4128231813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":4128231813}]},"public_addr":"172.21.15.90:6801/4128231813","cluster_addr":"172.21.15.90:6802/4128231813","heartbeat_back_addr":"172.21.15.90:6807/4128231813","heartbeat_front_addr":"172.21.15.90:6806/4128231813","state":["exists","up"]},{"osd":1,"uuid":"e81e8b79-a326-41e5-9f97-1da36297e891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6803","nonce":55615551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":55615551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":55615551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":55615551}]},"public_addr":"172.21.15.90:6803/55615551","cluster_addr":"172.21.15.90:6804/55615551","heartbeat_back_addr":"172.21.15.90:6805/55615551","heartbeat_front_addr":"172.21.15.90:6800/55615551","state":["exists","up"]},{"osd":2,"uuid":"a798cb89-96c0-4222-b32e-d08a97970e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":474973433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":474973433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6811","nonce":474973433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":474973433}]},"public_addr":"172.21.15.113:6807/474973433","cluster_addr":"172.21.15.113:6806/474973433","heartbeat_back_addr":"172.21.15.113:6811/474973433","heartbeat_front_addr":"172.21.15.113:6810/474973433","state":["exists","up"]},{"osd":3,"uuid":"446df045-a76b-4937-ae3f-dc84a4e1ca87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6805","nonce":2075119886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":2075119886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":2075119886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6803","nonce":2075119886}]},"public_addr":"172.21.15.113:6805/2075119886","cluster_addr":"172.21.15.113:6804/2075119886","heartbeat_back_addr":"172.21.15.113:6808/2075119886","heartbeat_front_addr":"172.21.15.113:6803/2075119886","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T01:59:50.079 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-23T01:59:50.080 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-23T01:59:50.080 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-23T01:59:50.080 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-23T01:59:50.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:50.189+0000 7fc40519d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:50.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:50.189+0000 7fb70239d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:50.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:50.189+0000 7f949f06d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:50.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:50.189+0000 7f621755d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:50.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:50.198+0000 7fc40519d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:50.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:50.198+0000 7fb70239d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:50.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:50.199+0000 7f949f06d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:50.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:50.200+0000 7f621755d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:50.234 INFO:teuthology.orchestra.run.smithi090.stdout:38654706546 2024-04-23T01:59:50.234 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-23T01:59:50.235 INFO:teuthology.orchestra.run.smithi090.stdout:38654706546 2024-04-23T01:59:50.236 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-23T01:59:50.239 INFO:teuthology.orchestra.run.smithi090.stdout:38654706546 2024-04-23T01:59:50.239 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-23T01:59:50.240 INFO:teuthology.orchestra.run.smithi090.stdout:38654706546 2024-04-23T01:59:50.241 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-23T01:59:50.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:50.345+0000 7f1661a06640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:50.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:50.359+0000 7f1661a06640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:50.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:50.388+0000 7f05faa6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:50.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:50.390+0000 7fa05c82d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:50.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:50.392+0000 7fb56a46d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:50.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:50.399+0000 7f05faa6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:50.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:50.400+0000 7fa05c82d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:50.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:50.402+0000 7fb56a46d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:50.574 INFO:teuthology.orchestra.run.smithi090.stdout:38654706542 2024-04-23T01:59:50.589 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654706546 got 38654706542 for osd.3 2024-04-23T01:59:50.604 INFO:teuthology.orchestra.run.smithi090.stdout:38654706542 2024-04-23T01:59:50.618 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654706546 got 38654706542 for osd.1 2024-04-23T01:59:50.648 INFO:teuthology.orchestra.run.smithi090.stdout:38654706542 2024-04-23T01:59:50.649 INFO:teuthology.orchestra.run.smithi090.stdout:38654706543 2024-04-23T01:59:50.662 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654706546 got 38654706542 for osd.0 2024-04-23T01:59:50.663 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654706546 got 38654706543 for osd.2 2024-04-23T01:59:51.589 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-23T01:59:51.619 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-23T01:59:51.663 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-23T01:59:51.663 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-23T01:59:51.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:51.694+0000 7f10aa19d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:51.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:51.705+0000 7f10aa19d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:51.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:51.740+0000 7f9f6f0ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:51.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:51.750+0000 7f9f6f0ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:51.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:51.785+0000 7fd9bb220640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:51.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:51.794+0000 7fd9bb220640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:51.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:51.791+0000 7fadc622d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:51.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:51.811+0000 7fadc622d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:51.917 INFO:teuthology.orchestra.run.smithi090.stdout:38654706542 2024-04-23T01:59:51.935 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654706546 got 38654706542 for osd.3 2024-04-23T01:59:51.971 INFO:teuthology.orchestra.run.smithi090.stdout:38654706542 2024-04-23T01:59:51.984 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654706546 got 38654706542 for osd.1 2024-04-23T01:59:52.007 INFO:teuthology.orchestra.run.smithi090.stdout:38654706543 2024-04-23T01:59:52.019 INFO:teuthology.orchestra.run.smithi090.stdout:38654706542 2024-04-23T01:59:52.020 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654706546 got 38654706543 for osd.2 2024-04-23T01:59:52.033 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654706546 got 38654706542 for osd.0 2024-04-23T01:59:52.936 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-23T01:59:52.985 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-23T01:59:53.021 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-23T01:59:53.034 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-23T01:59:53.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:53.049+0000 7f50dc95d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:53.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:53.059+0000 7f50dc95d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:53.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:53.097+0000 7fdc87e17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:53.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:53.109+0000 7fdc87e17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:53.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:53.144+0000 7f34c375d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:53.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:53.154+0000 7f34c375d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:53.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:53.167+0000 7fd21499d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:53.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:53.177+0000 7fd21499d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:53.268 INFO:teuthology.orchestra.run.smithi090.stdout:38654706546 2024-04-23T01:59:53.281 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654706546 got 38654706546 for osd.3 2024-04-23T01:59:53.282 DEBUG:teuthology.parallel:result is None 2024-04-23T01:59:53.351 INFO:teuthology.orchestra.run.smithi090.stdout:38654706546 2024-04-23T01:59:53.364 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654706546 got 38654706546 for osd.1 2024-04-23T01:59:53.365 DEBUG:teuthology.parallel:result is None 2024-04-23T01:59:53.378 INFO:teuthology.orchestra.run.smithi090.stdout:38654706547 2024-04-23T01:59:53.383 INFO:teuthology.orchestra.run.smithi090.stdout:38654706546 2024-04-23T01:59:53.391 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654706546 got 38654706547 for osd.0 2024-04-23T01:59:53.392 DEBUG:teuthology.parallel:result is None 2024-04-23T01:59:53.396 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654706546 got 38654706546 for osd.2 2024-04-23T01:59:53.396 DEBUG:teuthology.parallel:result is None 2024-04-23T01:59:53.397 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-23T01:59:53.397 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:59:53.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:53.540+0000 7f6fd306d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:53.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:53.547+0000 7f6fd306d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:53.739 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:59:53.740 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:59:53.752 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":543,"stamp":"2024-04-23T01:59:53.722745+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":483732,"kb_used_data":2444,"kb_used_omap":0,"kb_used_meta":480704,"kb_avail":377003628,"statfs":{"total":386547056640,"available":386051715072,"internally_reserved":0,"allocated":2502656,"data_stored":2642985,"data_compressed":89859,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":492240896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.293251"},"pg_stats":[{"pgid":"1.0","version":"12'97","reported_seq":383,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.813209+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:59:50.813209+0000","last_peered":"2024-04-23T01:59:50.813209+0000","last_clean":"2024-04-23T01:59:50.813209+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:59:50.813209+0000","last_undegraded":"2024-04-23T01:59:50.813209+0000","last_fullsized":"2024-04-23T01:59:50.813209+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":242,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.907953+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:59:50.907953+0000","last_peered":"2024-04-23T01:59:50.907953+0000","last_clean":"2024-04-23T01:59:50.907953+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:59:50.907953+0000","last_undegraded":"2024-04-23T01:59:50.907953+0000","last_fullsized":"2024-04-23T01:59:50.907953+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":250,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.825653+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:59:50.825653+0000","last_peered":"2024-04-23T01:59:50.825653+0000","last_clean":"2024-04-23T01:59:50.825653+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:59:50.825653+0000","last_undegraded":"2024-04-23T01:59:50.825653+0000","last_fullsized":"2024-04-23T01:59:50.825653+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":242,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.874501+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:59:50.874501+0000","last_peered":"2024-04-23T01:59:50.874501+0000","last_clean":"2024-04-23T01:59:50.874501+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:59:50.874501+0000","last_undegraded":"2024-04-23T01:59:50.874501+0000","last_fullsized":"2024-04-23T01:59:50.874501+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":250,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.847705+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:59:50.847705+0000","last_peered":"2024-04-23T01:59:50.847705+0000","last_clean":"2024-04-23T01:59:50.847705+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:59:50.847705+0000","last_undegraded":"2024-04-23T01:59:50.847705+0000","last_fullsized":"2024-04-23T01:59:50.847705+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":248,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:49.469154+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:59:49.469154+0000","last_peered":"2024-04-23T01:59:49.469154+0000","last_clean":"2024-04-23T01:59:49.469154+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:59:49.469154+0000","last_undegraded":"2024-04-23T01:59:49.469154+0000","last_fullsized":"2024-04-23T01:59:49.469154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":250,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.828388+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:59:50.828388+0000","last_peered":"2024-04-23T01:59:50.828388+0000","last_clean":"2024-04-23T01:59:50.828388+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:59:50.828388+0000","last_undegraded":"2024-04-23T01:59:50.828388+0000","last_fullsized":"2024-04-23T01:59:50.828388+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":248,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:49.468435+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:59:49.468435+0000","last_peered":"2024-04-23T01:59:49.468435+0000","last_clean":"2024-04-23T01:59:49.468435+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:59:49.468435+0000","last_undegraded":"2024-04-23T01:59:49.468435+0000","last_fullsized":"2024-04-23T01:59:49.468435+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":242,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.873519+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:59:50.873519+0000","last_peered":"2024-04-23T01:59:50.873519+0000","last_clean":"2024-04-23T01:59:50.873519+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:59:50.873519+0000","last_undegraded":"2024-04-23T01:59:50.873519+0000","last_fullsized":"2024-04-23T01:59:50.873519+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":126016,"kb_used_data":552,"kb_used_omap":0,"kb_used_meta":125376,"kb_avail":94245824,"statfs":{"total":96636764160,"available":96507723776,"internally_reserved":0,"allocated":565248,"data_stored":777501,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":128385024},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":122168,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":121216,"kb_avail":94249672,"statfs":{"total":96636764160,"available":96511664128,"internally_reserved":0,"allocated":655360,"data_stored":814189,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":124125184},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":117868,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":117056,"kb_avail":94253972,"statfs":{"total":96636764160,"available":96516067328,"internally_reserved":0,"allocated":716800,"data_stored":273794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":119865344},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":117680,"kb_used_data":552,"kb_used_omap":0,"kb_used_meta":117056,"kb_avail":94254160,"statfs":{"total":96636764160,"available":96516259840,"internally_reserved":0,"allocated":565248,"data_stored":777501,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":119865344},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:59:53.753 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:59:53.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:53.854+0000 7f9ae7c17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:53.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:53.861+0000 7f9ae7c17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:54.050 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:59:54.051 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:59:54.062 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":543,"stamp":"2024-04-23T01:59:53.722745+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":483732,"kb_used_data":2444,"kb_used_omap":0,"kb_used_meta":480704,"kb_avail":377003628,"statfs":{"total":386547056640,"available":386051715072,"internally_reserved":0,"allocated":2502656,"data_stored":2642985,"data_compressed":89859,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":492240896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.293251"},"pg_stats":[{"pgid":"1.0","version":"12'97","reported_seq":383,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.813209+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:59:50.813209+0000","last_peered":"2024-04-23T01:59:50.813209+0000","last_clean":"2024-04-23T01:59:50.813209+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:59:50.813209+0000","last_undegraded":"2024-04-23T01:59:50.813209+0000","last_fullsized":"2024-04-23T01:59:50.813209+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":242,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.907953+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:59:50.907953+0000","last_peered":"2024-04-23T01:59:50.907953+0000","last_clean":"2024-04-23T01:59:50.907953+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:59:50.907953+0000","last_undegraded":"2024-04-23T01:59:50.907953+0000","last_fullsized":"2024-04-23T01:59:50.907953+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":250,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.825653+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:59:50.825653+0000","last_peered":"2024-04-23T01:59:50.825653+0000","last_clean":"2024-04-23T01:59:50.825653+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:59:50.825653+0000","last_undegraded":"2024-04-23T01:59:50.825653+0000","last_fullsized":"2024-04-23T01:59:50.825653+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":242,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.874501+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:59:50.874501+0000","last_peered":"2024-04-23T01:59:50.874501+0000","last_clean":"2024-04-23T01:59:50.874501+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:59:50.874501+0000","last_undegraded":"2024-04-23T01:59:50.874501+0000","last_fullsized":"2024-04-23T01:59:50.874501+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":250,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.847705+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:59:50.847705+0000","last_peered":"2024-04-23T01:59:50.847705+0000","last_clean":"2024-04-23T01:59:50.847705+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:59:50.847705+0000","last_undegraded":"2024-04-23T01:59:50.847705+0000","last_fullsized":"2024-04-23T01:59:50.847705+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":248,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:49.469154+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:59:49.469154+0000","last_peered":"2024-04-23T01:59:49.469154+0000","last_clean":"2024-04-23T01:59:49.469154+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:59:49.469154+0000","last_undegraded":"2024-04-23T01:59:49.469154+0000","last_fullsized":"2024-04-23T01:59:49.469154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":250,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.828388+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:59:50.828388+0000","last_peered":"2024-04-23T01:59:50.828388+0000","last_clean":"2024-04-23T01:59:50.828388+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:59:50.828388+0000","last_undegraded":"2024-04-23T01:59:50.828388+0000","last_fullsized":"2024-04-23T01:59:50.828388+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":248,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:49.468435+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:59:49.468435+0000","last_peered":"2024-04-23T01:59:49.468435+0000","last_clean":"2024-04-23T01:59:49.468435+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:59:49.468435+0000","last_undegraded":"2024-04-23T01:59:49.468435+0000","last_fullsized":"2024-04-23T01:59:49.468435+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":242,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.873519+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:59:50.873519+0000","last_peered":"2024-04-23T01:59:50.873519+0000","last_clean":"2024-04-23T01:59:50.873519+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:59:50.873519+0000","last_undegraded":"2024-04-23T01:59:50.873519+0000","last_fullsized":"2024-04-23T01:59:50.873519+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":126016,"kb_used_data":552,"kb_used_omap":0,"kb_used_meta":125376,"kb_avail":94245824,"statfs":{"total":96636764160,"available":96507723776,"internally_reserved":0,"allocated":565248,"data_stored":777501,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":128385024},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":122168,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":121216,"kb_avail":94249672,"statfs":{"total":96636764160,"available":96511664128,"internally_reserved":0,"allocated":655360,"data_stored":814189,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":124125184},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":117868,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":117056,"kb_avail":94253972,"statfs":{"total":96636764160,"available":96516067328,"internally_reserved":0,"allocated":716800,"data_stored":273794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":119865344},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":117680,"kb_used_data":552,"kb_used_omap":0,"kb_used_meta":117056,"kb_avail":94254160,"statfs":{"total":96636764160,"available":96516259840,"internally_reserved":0,"allocated":565248,"data_stored":777501,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":119865344},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:59:54.063 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-23T01:59:54.063 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-04-23T01:59:54.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:54.164+0000 7f0ade06d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:54.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:54.174+0000 7f0ade06d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:54.385 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-23T01:59:54.394 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-23T01:59:54.395 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:59:54.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:54.497+0000 7fc1412ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:54.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:54.504+0000 7fc1412ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:54.697 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:59:54.697 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:59:54.709 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":543,"stamp":"2024-04-23T01:59:53.722745+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":483732,"kb_used_data":2444,"kb_used_omap":0,"kb_used_meta":480704,"kb_avail":377003628,"statfs":{"total":386547056640,"available":386051715072,"internally_reserved":0,"allocated":2502656,"data_stored":2642985,"data_compressed":89859,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":492240896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.293251"},"pg_stats":[{"pgid":"1.0","version":"12'97","reported_seq":383,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.813209+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:59:50.813209+0000","last_peered":"2024-04-23T01:59:50.813209+0000","last_clean":"2024-04-23T01:59:50.813209+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:59:50.813209+0000","last_undegraded":"2024-04-23T01:59:50.813209+0000","last_fullsized":"2024-04-23T01:59:50.813209+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":242,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.907953+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:59:50.907953+0000","last_peered":"2024-04-23T01:59:50.907953+0000","last_clean":"2024-04-23T01:59:50.907953+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:59:50.907953+0000","last_undegraded":"2024-04-23T01:59:50.907953+0000","last_fullsized":"2024-04-23T01:59:50.907953+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":250,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.825653+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:59:50.825653+0000","last_peered":"2024-04-23T01:59:50.825653+0000","last_clean":"2024-04-23T01:59:50.825653+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:59:50.825653+0000","last_undegraded":"2024-04-23T01:59:50.825653+0000","last_fullsized":"2024-04-23T01:59:50.825653+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":242,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.874501+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:59:50.874501+0000","last_peered":"2024-04-23T01:59:50.874501+0000","last_clean":"2024-04-23T01:59:50.874501+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:59:50.874501+0000","last_undegraded":"2024-04-23T01:59:50.874501+0000","last_fullsized":"2024-04-23T01:59:50.874501+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":250,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.847705+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:59:50.847705+0000","last_peered":"2024-04-23T01:59:50.847705+0000","last_clean":"2024-04-23T01:59:50.847705+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:59:50.847705+0000","last_undegraded":"2024-04-23T01:59:50.847705+0000","last_fullsized":"2024-04-23T01:59:50.847705+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":248,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:49.469154+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:59:49.469154+0000","last_peered":"2024-04-23T01:59:49.469154+0000","last_clean":"2024-04-23T01:59:49.469154+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:59:49.469154+0000","last_undegraded":"2024-04-23T01:59:49.469154+0000","last_fullsized":"2024-04-23T01:59:49.469154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":250,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.828388+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:59:50.828388+0000","last_peered":"2024-04-23T01:59:50.828388+0000","last_clean":"2024-04-23T01:59:50.828388+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:59:50.828388+0000","last_undegraded":"2024-04-23T01:59:50.828388+0000","last_fullsized":"2024-04-23T01:59:50.828388+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":248,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:49.468435+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:59:49.468435+0000","last_peered":"2024-04-23T01:59:49.468435+0000","last_clean":"2024-04-23T01:59:49.468435+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:59:49.468435+0000","last_undegraded":"2024-04-23T01:59:49.468435+0000","last_fullsized":"2024-04-23T01:59:49.468435+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":242,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.873519+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:59:50.873519+0000","last_peered":"2024-04-23T01:59:50.873519+0000","last_clean":"2024-04-23T01:59:50.873519+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:59:50.873519+0000","last_undegraded":"2024-04-23T01:59:50.873519+0000","last_fullsized":"2024-04-23T01:59:50.873519+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":126016,"kb_used_data":552,"kb_used_omap":0,"kb_used_meta":125376,"kb_avail":94245824,"statfs":{"total":96636764160,"available":96507723776,"internally_reserved":0,"allocated":565248,"data_stored":777501,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":128385024},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":122168,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":121216,"kb_avail":94249672,"statfs":{"total":96636764160,"available":96511664128,"internally_reserved":0,"allocated":655360,"data_stored":814189,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":124125184},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":117868,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":117056,"kb_avail":94253972,"statfs":{"total":96636764160,"available":96516067328,"internally_reserved":0,"allocated":716800,"data_stored":273794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":119865344},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":117680,"kb_used_data":552,"kb_used_omap":0,"kb_used_meta":117056,"kb_avail":94254160,"statfs":{"total":96636764160,"available":96516259840,"internally_reserved":0,"allocated":565248,"data_stored":777501,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":119865344},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:59:54.710 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:59:54.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:54.811+0000 7f0dfc75d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:54.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:54.819+0000 7f0dfc75d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:55.011 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:59:55.011 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:59:55.022 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":543,"stamp":"2024-04-23T01:59:53.722745+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":483732,"kb_used_data":2444,"kb_used_omap":0,"kb_used_meta":480704,"kb_avail":377003628,"statfs":{"total":386547056640,"available":386051715072,"internally_reserved":0,"allocated":2502656,"data_stored":2642985,"data_compressed":89859,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":492240896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.293251"},"pg_stats":[{"pgid":"1.0","version":"12'97","reported_seq":383,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.813209+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:59:50.813209+0000","last_peered":"2024-04-23T01:59:50.813209+0000","last_clean":"2024-04-23T01:59:50.813209+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:59:50.813209+0000","last_undegraded":"2024-04-23T01:59:50.813209+0000","last_fullsized":"2024-04-23T01:59:50.813209+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":242,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.907953+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:59:50.907953+0000","last_peered":"2024-04-23T01:59:50.907953+0000","last_clean":"2024-04-23T01:59:50.907953+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:59:50.907953+0000","last_undegraded":"2024-04-23T01:59:50.907953+0000","last_fullsized":"2024-04-23T01:59:50.907953+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":250,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.825653+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:59:50.825653+0000","last_peered":"2024-04-23T01:59:50.825653+0000","last_clean":"2024-04-23T01:59:50.825653+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:59:50.825653+0000","last_undegraded":"2024-04-23T01:59:50.825653+0000","last_fullsized":"2024-04-23T01:59:50.825653+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":242,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.874501+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:59:50.874501+0000","last_peered":"2024-04-23T01:59:50.874501+0000","last_clean":"2024-04-23T01:59:50.874501+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:59:50.874501+0000","last_undegraded":"2024-04-23T01:59:50.874501+0000","last_fullsized":"2024-04-23T01:59:50.874501+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":250,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.847705+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:59:50.847705+0000","last_peered":"2024-04-23T01:59:50.847705+0000","last_clean":"2024-04-23T01:59:50.847705+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:59:50.847705+0000","last_undegraded":"2024-04-23T01:59:50.847705+0000","last_fullsized":"2024-04-23T01:59:50.847705+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":248,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:49.469154+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:59:49.469154+0000","last_peered":"2024-04-23T01:59:49.469154+0000","last_clean":"2024-04-23T01:59:49.469154+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:59:49.469154+0000","last_undegraded":"2024-04-23T01:59:49.469154+0000","last_fullsized":"2024-04-23T01:59:49.469154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":250,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.828388+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:59:50.828388+0000","last_peered":"2024-04-23T01:59:50.828388+0000","last_clean":"2024-04-23T01:59:50.828388+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:59:50.828388+0000","last_undegraded":"2024-04-23T01:59:50.828388+0000","last_fullsized":"2024-04-23T01:59:50.828388+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":248,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:49.468435+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:59:49.468435+0000","last_peered":"2024-04-23T01:59:49.468435+0000","last_clean":"2024-04-23T01:59:49.468435+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:59:49.468435+0000","last_undegraded":"2024-04-23T01:59:49.468435+0000","last_fullsized":"2024-04-23T01:59:49.468435+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":242,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.873519+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:59:50.873519+0000","last_peered":"2024-04-23T01:59:50.873519+0000","last_clean":"2024-04-23T01:59:50.873519+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:59:50.873519+0000","last_undegraded":"2024-04-23T01:59:50.873519+0000","last_fullsized":"2024-04-23T01:59:50.873519+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":126016,"kb_used_data":552,"kb_used_omap":0,"kb_used_meta":125376,"kb_avail":94245824,"statfs":{"total":96636764160,"available":96507723776,"internally_reserved":0,"allocated":565248,"data_stored":777501,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":128385024},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":122168,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":121216,"kb_avail":94249672,"statfs":{"total":96636764160,"available":96511664128,"internally_reserved":0,"allocated":655360,"data_stored":814189,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":124125184},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":117868,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":117056,"kb_avail":94253972,"statfs":{"total":96636764160,"available":96516067328,"internally_reserved":0,"allocated":716800,"data_stored":273794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":119865344},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":117680,"kb_used_data":552,"kb_used_omap":0,"kb_used_meta":117056,"kb_avail":94254160,"statfs":{"total":96636764160,"available":96516259840,"internally_reserved":0,"allocated":565248,"data_stored":777501,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":119865344},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:59:55.023 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-23T01:59:55.023 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:59:55.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:55.124+0000 7fbfa802d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:55.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:55.131+0000 7fbfa802d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:55.320 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:59:55.321 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:59:55.332 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":543,"stamp":"2024-04-23T01:59:53.722745+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":483732,"kb_used_data":2444,"kb_used_omap":0,"kb_used_meta":480704,"kb_avail":377003628,"statfs":{"total":386547056640,"available":386051715072,"internally_reserved":0,"allocated":2502656,"data_stored":2642985,"data_compressed":89859,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":492240896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.293251"},"pg_stats":[{"pgid":"1.0","version":"12'97","reported_seq":383,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.813209+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:59:50.813209+0000","last_peered":"2024-04-23T01:59:50.813209+0000","last_clean":"2024-04-23T01:59:50.813209+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:59:50.813209+0000","last_undegraded":"2024-04-23T01:59:50.813209+0000","last_fullsized":"2024-04-23T01:59:50.813209+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":242,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.907953+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:59:50.907953+0000","last_peered":"2024-04-23T01:59:50.907953+0000","last_clean":"2024-04-23T01:59:50.907953+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:59:50.907953+0000","last_undegraded":"2024-04-23T01:59:50.907953+0000","last_fullsized":"2024-04-23T01:59:50.907953+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":250,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.825653+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:59:50.825653+0000","last_peered":"2024-04-23T01:59:50.825653+0000","last_clean":"2024-04-23T01:59:50.825653+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:59:50.825653+0000","last_undegraded":"2024-04-23T01:59:50.825653+0000","last_fullsized":"2024-04-23T01:59:50.825653+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":242,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.874501+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:59:50.874501+0000","last_peered":"2024-04-23T01:59:50.874501+0000","last_clean":"2024-04-23T01:59:50.874501+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:59:50.874501+0000","last_undegraded":"2024-04-23T01:59:50.874501+0000","last_fullsized":"2024-04-23T01:59:50.874501+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":250,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.847705+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:59:50.847705+0000","last_peered":"2024-04-23T01:59:50.847705+0000","last_clean":"2024-04-23T01:59:50.847705+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:59:50.847705+0000","last_undegraded":"2024-04-23T01:59:50.847705+0000","last_fullsized":"2024-04-23T01:59:50.847705+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":248,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:49.469154+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:59:49.469154+0000","last_peered":"2024-04-23T01:59:49.469154+0000","last_clean":"2024-04-23T01:59:49.469154+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:59:49.469154+0000","last_undegraded":"2024-04-23T01:59:49.469154+0000","last_fullsized":"2024-04-23T01:59:49.469154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":250,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.828388+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:59:50.828388+0000","last_peered":"2024-04-23T01:59:50.828388+0000","last_clean":"2024-04-23T01:59:50.828388+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:59:50.828388+0000","last_undegraded":"2024-04-23T01:59:50.828388+0000","last_fullsized":"2024-04-23T01:59:50.828388+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":248,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:49.468435+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:59:49.468435+0000","last_peered":"2024-04-23T01:59:49.468435+0000","last_clean":"2024-04-23T01:59:49.468435+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:59:49.468435+0000","last_undegraded":"2024-04-23T01:59:49.468435+0000","last_fullsized":"2024-04-23T01:59:49.468435+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":242,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.873519+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:59:50.873519+0000","last_peered":"2024-04-23T01:59:50.873519+0000","last_clean":"2024-04-23T01:59:50.873519+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:59:50.873519+0000","last_undegraded":"2024-04-23T01:59:50.873519+0000","last_fullsized":"2024-04-23T01:59:50.873519+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":126016,"kb_used_data":552,"kb_used_omap":0,"kb_used_meta":125376,"kb_avail":94245824,"statfs":{"total":96636764160,"available":96507723776,"internally_reserved":0,"allocated":565248,"data_stored":777501,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":128385024},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":122168,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":121216,"kb_avail":94249672,"statfs":{"total":96636764160,"available":96511664128,"internally_reserved":0,"allocated":655360,"data_stored":814189,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":124125184},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":117868,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":117056,"kb_avail":94253972,"statfs":{"total":96636764160,"available":96516067328,"internally_reserved":0,"allocated":716800,"data_stored":273794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":119865344},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":117680,"kb_used_data":552,"kb_used_omap":0,"kb_used_meta":117056,"kb_avail":94254160,"statfs":{"total":96636764160,"available":96516259840,"internally_reserved":0,"allocated":565248,"data_stored":777501,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":119865344},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:59:55.333 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T01:59:55.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:55.432+0000 7ff53ab5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:55.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:55.440+0000 7ff53ab5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:55.630 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:59:55.631 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":106,"fsid":"0a731bfd-0fc4-4c1b-8e4b-f6dfbc059bcc","created":"2024-04-23T01:44:59.145590+0000","modified":"2024-04-23T01:59:49.435223+0000","last_up_change":"2024-04-23T01:45:06.551915+0000","last_in_change":"2024-04-23T01:45:01.284163+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T01:45:07.564959+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-23T01:45:09.460600+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93e81b79-2e2e-4e90-ac13-13a26b240245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6801","nonce":4128231813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":4128231813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6807","nonce":4128231813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":4128231813}]},"public_addr":"172.21.15.90:6801/4128231813","cluster_addr":"172.21.15.90:6802/4128231813","heartbeat_back_addr":"172.21.15.90:6807/4128231813","heartbeat_front_addr":"172.21.15.90:6806/4128231813","state":["exists","up"]},{"osd":1,"uuid":"e81e8b79-a326-41e5-9f97-1da36297e891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6803","nonce":55615551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":55615551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":55615551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":55615551}]},"public_addr":"172.21.15.90:6803/55615551","cluster_addr":"172.21.15.90:6804/55615551","heartbeat_back_addr":"172.21.15.90:6805/55615551","heartbeat_front_addr":"172.21.15.90:6800/55615551","state":["exists","up"]},{"osd":2,"uuid":"a798cb89-96c0-4222-b32e-d08a97970e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":474973433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":474973433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6811","nonce":474973433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":474973433}]},"public_addr":"172.21.15.113:6807/474973433","cluster_addr":"172.21.15.113:6806/474973433","heartbeat_back_addr":"172.21.15.113:6811/474973433","heartbeat_front_addr":"172.21.15.113:6810/474973433","state":["exists","up"]},{"osd":3,"uuid":"446df045-a76b-4937-ae3f-dc84a4e1ca87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6805","nonce":2075119886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":2075119886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":2075119886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6803","nonce":2075119886}]},"public_addr":"172.21.15.113:6805/2075119886","cluster_addr":"172.21.15.113:6804/2075119886","heartbeat_back_addr":"172.21.15.113:6808/2075119886","heartbeat_front_addr":"172.21.15.113:6803/2075119886","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T01:59:56.643 INFO:tasks.ceph:Scrubbing osd.0 2024-04-23T01:59:56.644 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-23T01:59:56.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:56.754+0000 7f9a6986d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:56.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:56.766+0000 7f9a6986d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:56.783 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-23T01:59:56.783 INFO:teuthology.orchestra.run.smithi090.stdout: "success": "" 2024-04-23T01:59:56.783 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-23T01:59:56.794 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-23T01:59:56.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:56.894+0000 7f0bbb06d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:56.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:56.903+0000 7f0bbb06d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:57.098 INFO:teuthology.orchestra.run.smithi090.stderr:instructed osd(s) 0 to deep-scrub 2024-04-23T01:59:57.115 INFO:tasks.ceph:Scrubbing osd.1 2024-04-23T01:59:57.115 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-23T01:59:57.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:57.215+0000 7f172c82d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:57.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:57.227+0000 7f172c82d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:57.244 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-23T01:59:57.244 INFO:teuthology.orchestra.run.smithi090.stdout: "success": "" 2024-04-23T01:59:57.244 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-23T01:59:57.255 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-23T01:59:57.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:57.356+0000 7f743662d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:57.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:57.364+0000 7f743662d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:57.555 INFO:teuthology.orchestra.run.smithi090.stderr:instructed osd(s) 1 to deep-scrub 2024-04-23T01:59:57.574 INFO:tasks.ceph:Scrubbing osd.2 2024-04-23T01:59:57.574 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-23T01:59:57.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:57.674+0000 7f593ea6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:57.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:57.686+0000 7f593ea6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:57.703 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-23T01:59:57.703 INFO:teuthology.orchestra.run.smithi090.stdout: "success": "" 2024-04-23T01:59:57.703 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-23T01:59:57.714 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-23T01:59:57.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:57.813+0000 7fdd48024640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:57.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:57.822+0000 7fdd48024640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:58.013 INFO:teuthology.orchestra.run.smithi090.stderr:instructed osd(s) 2 to deep-scrub 2024-04-23T01:59:58.027 INFO:tasks.ceph:Scrubbing osd.3 2024-04-23T01:59:58.027 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-23T01:59:58.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:58.127+0000 7fd5ee62d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:58.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:58.139+0000 7fd5ee62d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:58.158 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-23T01:59:58.158 INFO:teuthology.orchestra.run.smithi090.stdout: "success": "" 2024-04-23T01:59:58.158 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-23T01:59:58.169 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-23T01:59:58.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:58.269+0000 7f004a2ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:58.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:58.278+0000 7f004a2ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:58.467 INFO:teuthology.orchestra.run.smithi090.stderr:instructed osd(s) 3 to deep-scrub 2024-04-23T01:59:58.483 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T01:59:58.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:58.583+0000 7fa52706d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:58.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T01:59:58.590+0000 7fa52706d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T01:59:58.780 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T01:59:58.781 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T01:59:58.793 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":545,"stamp":"2024-04-23T01:59:57.723759+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":483732,"kb_used_data":2444,"kb_used_omap":0,"kb_used_meta":480704,"kb_avail":377003628,"statfs":{"total":386547056640,"available":386051715072,"internally_reserved":0,"allocated":2502656,"data_stored":2642985,"data_compressed":89859,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":492240896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.287896"},"pg_stats":[{"pgid":"1.0","version":"12'97","reported_seq":383,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.813209+0000","last_change":"2024-04-23T01:45:08.674997+0000","last_active":"2024-04-23T01:59:50.813209+0000","last_peered":"2024-04-23T01:59:50.813209+0000","last_clean":"2024-04-23T01:59:50.813209+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:59:50.813209+0000","last_undegraded":"2024-04-23T01:59:50.813209+0000","last_fullsized":"2024-04-23T01:59:50.813209+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:07.615161+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":242,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.907953+0000","last_change":"2024-04-23T01:45:10.804913+0000","last_active":"2024-04-23T01:59:50.907953+0000","last_peered":"2024-04-23T01:59:50.907953+0000","last_clean":"2024-04-23T01:59:50.907953+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:59:50.907953+0000","last_undegraded":"2024-04-23T01:59:50.907953+0000","last_fullsized":"2024-04-23T01:59:50.907953+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":250,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.825653+0000","last_change":"2024-04-23T01:45:10.771182+0000","last_active":"2024-04-23T01:59:50.825653+0000","last_peered":"2024-04-23T01:59:50.825653+0000","last_clean":"2024-04-23T01:59:50.825653+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:59:50.825653+0000","last_undegraded":"2024-04-23T01:59:50.825653+0000","last_fullsized":"2024-04-23T01:59:50.825653+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":242,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.874501+0000","last_change":"2024-04-23T01:45:10.756063+0000","last_active":"2024-04-23T01:59:50.874501+0000","last_peered":"2024-04-23T01:59:50.874501+0000","last_clean":"2024-04-23T01:59:50.874501+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:59:50.874501+0000","last_undegraded":"2024-04-23T01:59:50.874501+0000","last_fullsized":"2024-04-23T01:59:50.874501+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":250,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.847705+0000","last_change":"2024-04-23T01:45:10.784679+0000","last_active":"2024-04-23T01:59:50.847705+0000","last_peered":"2024-04-23T01:59:50.847705+0000","last_clean":"2024-04-23T01:59:50.847705+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:59:50.847705+0000","last_undegraded":"2024-04-23T01:59:50.847705+0000","last_fullsized":"2024-04-23T01:59:50.847705+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":248,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:49.469154+0000","last_change":"2024-04-23T01:45:10.767135+0000","last_active":"2024-04-23T01:59:49.469154+0000","last_peered":"2024-04-23T01:59:49.469154+0000","last_clean":"2024-04-23T01:59:49.469154+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:59:49.469154+0000","last_undegraded":"2024-04-23T01:59:49.469154+0000","last_fullsized":"2024-04-23T01:59:49.469154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":250,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.828388+0000","last_change":"2024-04-23T01:45:10.761664+0000","last_active":"2024-04-23T01:59:50.828388+0000","last_peered":"2024-04-23T01:59:50.828388+0000","last_clean":"2024-04-23T01:59:50.828388+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:59:50.828388+0000","last_undegraded":"2024-04-23T01:59:50.828388+0000","last_fullsized":"2024-04-23T01:59:50.828388+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":248,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:49.468435+0000","last_change":"2024-04-23T01:45:10.769771+0000","last_active":"2024-04-23T01:59:49.468435+0000","last_peered":"2024-04-23T01:59:49.468435+0000","last_clean":"2024-04-23T01:59:49.468435+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:59:49.468435+0000","last_undegraded":"2024-04-23T01:59:49.468435+0000","last_fullsized":"2024-04-23T01:59:49.468435+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":242,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:50.873519+0000","last_change":"2024-04-23T01:45:10.780535+0000","last_active":"2024-04-23T01:59:50.873519+0000","last_peered":"2024-04-23T01:59:50.873519+0000","last_clean":"2024-04-23T01:59:50.873519+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:59:50.873519+0000","last_undegraded":"2024-04-23T01:59:50.873519+0000","last_fullsized":"2024-04-23T01:59:50.873519+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:45:09.667353+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706552,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":126016,"kb_used_data":552,"kb_used_omap":0,"kb_used_meta":125376,"kb_avail":94245824,"statfs":{"total":96636764160,"available":96507723776,"internally_reserved":0,"allocated":565248,"data_stored":777501,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":128385024},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706553,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":122168,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":121216,"kb_avail":94249672,"statfs":{"total":96636764160,"available":96511664128,"internally_reserved":0,"allocated":655360,"data_stored":814189,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":124125184},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706552,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":117868,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":117056,"kb_avail":94253972,"statfs":{"total":96636764160,"available":96516067328,"internally_reserved":0,"allocated":716800,"data_stored":273794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":119865344},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706552,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":117680,"kb_used_data":552,"kb_used_omap":0,"kb_used_meta":117056,"kb_avail":94254160,"statfs":{"total":96636764160,"available":96516259840,"internally_reserved":0,"allocated":565248,"data_stored":777501,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":119865344},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T01:59:58.796 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-23T01:45:07.615161+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=1, tm_min=45, tm_sec=7, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=1, tm_min=59, tm_sec=55, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T01:59:58.797 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-23T01:45:09.667353+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=1, tm_min=45, tm_sec=9, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=1, tm_min=59, tm_sec=55, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T01:59:58.797 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-23T01:45:09.667353+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=1, tm_min=45, tm_sec=9, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=1, tm_min=59, tm_sec=55, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T01:59:58.797 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-04-23T01:45:09.667353+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=1, tm_min=45, tm_sec=9, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=1, tm_min=59, tm_sec=55, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T01:59:58.797 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-23T01:45:09.667353+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=1, tm_min=45, tm_sec=9, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=1, tm_min=59, tm_sec=55, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T01:59:58.797 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-23T01:45:09.667353+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=1, tm_min=45, tm_sec=9, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=1, tm_min=59, tm_sec=55, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T01:59:58.797 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-23T01:45:09.667353+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=1, tm_min=45, tm_sec=9, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=1, tm_min=59, tm_sec=55, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T01:59:58.797 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-04-23T01:45:09.667353+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=1, tm_min=45, tm_sec=9, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=1, tm_min=59, tm_sec=55, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T01:59:58.797 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-23T01:45:09.667353+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=1, tm_min=45, tm_sec=9, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=1, tm_min=59, tm_sec=55, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T01:59:58.797 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-23T02:00:18.799 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T02:00:18.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T02:00:18.913+0000 7fe9c639d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T02:00:18.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T02:00:18.921+0000 7fe9c639d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T02:00:19.106 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-23T02:00:19.107 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-23T02:00:19.120 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":555,"stamp":"2024-04-23T02:00:17.728794+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":483732,"kb_used_data":2444,"kb_used_omap":0,"kb_used_meta":480704,"kb_avail":377003628,"statfs":{"total":386547056640,"available":386051715072,"internally_reserved":0,"allocated":2502656,"data_stored":2642985,"data_compressed":89859,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":492240896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003038"},"pg_stats":[{"pgid":"1.0","version":"12'97","reported_seq":385,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:58.468277+0000","last_change":"2024-04-23T01:59:58.492296+0000","last_active":"2024-04-23T01:59:58.468277+0000","last_peered":"2024-04-23T01:59:58.468277+0000","last_clean":"2024-04-23T01:59:58.468277+0000","last_became_active":"2024-04-23T01:45:08.669636+0000","last_became_peered":"2024-04-23T01:45:08.669636+0000","last_unstale":"2024-04-23T01:59:58.468277+0000","last_undegraded":"2024-04-23T01:59:58.468277+0000","last_fullsized":"2024-04-23T01:59:58.468277+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"12'97","last_scrub_stamp":"2024-04-23T01:59:58.492011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:59:58.492011+0000","last_clean_scrub_stamp":"2024-04-23T01:45:07.615161+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":244,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:57.558011+0000","last_change":"2024-04-23T01:59:57.571816+0000","last_active":"2024-04-23T01:59:57.558011+0000","last_peered":"2024-04-23T01:59:57.558011+0000","last_clean":"2024-04-23T01:59:57.558011+0000","last_became_active":"2024-04-23T01:45:10.799014+0000","last_became_peered":"2024-04-23T01:45:10.799014+0000","last_unstale":"2024-04-23T01:59:57.558011+0000","last_undegraded":"2024-04-23T01:59:57.558011+0000","last_fullsized":"2024-04-23T01:59:57.558011+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:59:57.571645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:59:57.571645+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":252,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:57.098405+0000","last_change":"2024-04-23T01:59:57.107475+0000","last_active":"2024-04-23T01:59:57.098405+0000","last_peered":"2024-04-23T01:59:57.098405+0000","last_clean":"2024-04-23T01:59:57.098405+0000","last_became_active":"2024-04-23T01:45:10.766553+0000","last_became_peered":"2024-04-23T01:45:10.766553+0000","last_unstale":"2024-04-23T01:59:57.098405+0000","last_undegraded":"2024-04-23T01:59:57.098405+0000","last_fullsized":"2024-04-23T01:59:57.098405+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:59:57.107286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:59:57.107286+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":244,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:57.555838+0000","last_change":"2024-04-23T01:59:57.568612+0000","last_active":"2024-04-23T01:59:57.555838+0000","last_peered":"2024-04-23T01:59:57.555838+0000","last_clean":"2024-04-23T01:59:57.555838+0000","last_became_active":"2024-04-23T01:45:10.747117+0000","last_became_peered":"2024-04-23T01:45:10.747117+0000","last_unstale":"2024-04-23T01:59:57.555838+0000","last_undegraded":"2024-04-23T01:59:57.555838+0000","last_fullsized":"2024-04-23T01:59:57.555838+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:59:57.568446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:59:57.568446+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":252,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:57.098776+0000","last_change":"2024-04-23T01:59:57.108347+0000","last_active":"2024-04-23T01:59:57.098776+0000","last_peered":"2024-04-23T01:59:57.098776+0000","last_clean":"2024-04-23T01:59:57.098776+0000","last_became_active":"2024-04-23T01:45:10.776581+0000","last_became_peered":"2024-04-23T01:45:10.776581+0000","last_unstale":"2024-04-23T01:59:57.098776+0000","last_undegraded":"2024-04-23T01:59:57.098776+0000","last_fullsized":"2024-04-23T01:59:57.098776+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:59:57.108065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:59:57.108065+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":250,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:58.014295+0000","last_change":"2024-04-23T01:59:58.024667+0000","last_active":"2024-04-23T01:59:58.014295+0000","last_peered":"2024-04-23T01:59:58.014295+0000","last_clean":"2024-04-23T01:59:58.014295+0000","last_became_active":"2024-04-23T01:45:10.762105+0000","last_became_peered":"2024-04-23T01:45:10.762105+0000","last_unstale":"2024-04-23T01:59:58.014295+0000","last_undegraded":"2024-04-23T01:59:58.014295+0000","last_fullsized":"2024-04-23T01:59:58.014295+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:59:58.024474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:59:58.024474+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":252,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:58.469753+0000","last_change":"2024-04-23T01:59:58.485175+0000","last_active":"2024-04-23T01:59:58.469753+0000","last_peered":"2024-04-23T01:59:58.469753+0000","last_clean":"2024-04-23T01:59:58.469753+0000","last_became_active":"2024-04-23T01:45:10.753884+0000","last_became_peered":"2024-04-23T01:45:10.753884+0000","last_unstale":"2024-04-23T01:59:58.469753+0000","last_undegraded":"2024-04-23T01:59:58.469753+0000","last_fullsized":"2024-04-23T01:59:58.469753+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:59:58.484818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:59:58.484818+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":250,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:58.014302+0000","last_change":"2024-04-23T01:59:58.023547+0000","last_active":"2024-04-23T01:59:58.014302+0000","last_peered":"2024-04-23T01:59:58.014302+0000","last_clean":"2024-04-23T01:59:58.014302+0000","last_became_active":"2024-04-23T01:45:10.764317+0000","last_became_peered":"2024-04-23T01:45:10.764317+0000","last_unstale":"2024-04-23T01:59:58.014302+0000","last_undegraded":"2024-04-23T01:59:58.014302+0000","last_fullsized":"2024-04-23T01:59:58.014302+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:59:58.023323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:59:58.023323+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":244,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-23T01:59:57.557033+0000","last_change":"2024-04-23T01:59:57.572680+0000","last_active":"2024-04-23T01:59:57.557033+0000","last_peered":"2024-04-23T01:59:57.557033+0000","last_clean":"2024-04-23T01:59:57.557033+0000","last_became_active":"2024-04-23T01:45:10.772407+0000","last_became_peered":"2024-04-23T01:45:10.772407+0000","last_unstale":"2024-04-23T01:59:57.557033+0000","last_undegraded":"2024-04-23T01:59:57.557033+0000","last_fullsized":"2024-04-23T01:59:57.557033+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T01:59:57.572521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T01:59:57.572521+0000","last_clean_scrub_stamp":"2024-04-23T01:45:09.667353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706572,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":126016,"kb_used_data":552,"kb_used_omap":0,"kb_used_meta":125376,"kb_avail":94245824,"statfs":{"total":96636764160,"available":96507723776,"internally_reserved":0,"allocated":565248,"data_stored":777501,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":128385024},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706573,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":122168,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":121216,"kb_avail":94249672,"statfs":{"total":96636764160,"available":96511664128,"internally_reserved":0,"allocated":655360,"data_stored":814189,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":124125184},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706572,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":117868,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":117056,"kb_avail":94253972,"statfs":{"total":96636764160,"available":96516067328,"internally_reserved":0,"allocated":716800,"data_stored":273794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":119865344},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706572,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":117680,"kb_used_data":552,"kb_used_omap":0,"kb_used_meta":117056,"kb_avail":94254160,"statfs":{"total":96636764160,"available":96516259840,"internally_reserved":0,"allocated":565248,"data_stored":777501,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":119865344},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":29953,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T02:00:19.121 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-23T02:00:19.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T02:00:19.219+0000 7f758ba6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T02:00:19.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-23T02:00:19.230+0000 7f758ba6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-23T02:00:19.442 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-23T02:00:19.442 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-23T02:00:19.442 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-23T02:00:19.442 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T02:00:19.542 INFO:tasks.ceph.osd.0:Stopped 2024-04-23T02:00:19.543 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-23T02:00:19.543 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T02:00:19.639 INFO:tasks.ceph.osd.1:Stopped 2024-04-23T02:00:19.639 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-23T02:00:19.639 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T02:00:19.734 INFO:tasks.ceph.osd.2:Stopped 2024-04-23T02:00:19.735 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-23T02:00:19.735 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T02:00:19.823 INFO:tasks.ceph.osd.3:Stopped 2024-04-23T02:00:19.823 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-23T02:00:19.823 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-23T02:00:19.823 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T02:00:19.850 INFO:tasks.ceph.mgr.x:Stopped 2024-04-23T02:00:19.850 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-23T02:00:19.851 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-23T02:00:19.851 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T02:00:19.863 INFO:tasks.ceph.mon.a:Stopped 2024-04-23T02:00:19.863 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-23T02:00:19.863 DEBUG:teuthology.orchestra.run.smithi090:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | head -n 1 2024-04-23T02:00:19.932 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi090.front.sepia.ceph.com 2024-04-23T02:00:19.932 DEBUG:teuthology.orchestra.run.smithi090:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-23T02:00:20.175 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi090.front.sepia.ceph.com 2024-04-23T02:00:20.176 DEBUG:teuthology.orchestra.run.smithi090:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-23T02:00:20.272 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi113.front.sepia.ceph.com 2024-04-23T02:00:20.272 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-23T02:00:20.434 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi113.front.sepia.ceph.com 2024-04-23T02:00:20.435 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-23T02:00:20.575 INFO:tasks.ceph:Archiving mon data... 2024-04-23T02:00:20.577 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yingxin-2024-04-23_01:23:55-crimson-rados-ci-yingxin-crimson-fix-tri-mutex-4-distro-default-smithi/7669459/data/mon.a.tgz 2024-04-23T02:00:20.577 DEBUG:teuthology.orchestra.run.smithi090:> mktemp 2024-04-23T02:00:20.592 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/tmp.qYw1EUGXdW 2024-04-23T02:00:20.592 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.qYw1EUGXdW 2024-04-23T02:00:20.728 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0666 /tmp/tmp.qYw1EUGXdW 2024-04-23T02:00:20.812 DEBUG:teuthology.orchestra.remote:smithi090:/tmp/tmp.qYw1EUGXdW is 236KB 2024-04-23T02:00:20.908 DEBUG:teuthology.orchestra.run.smithi090:> rm -fr /tmp/tmp.qYw1EUGXdW 2024-04-23T02:00:20.922 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-23T02:00:20.923 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-23T02:00:20.964 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-23T02:00:21.040 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-23T02:00:21.055 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-23T02:00:21.056 INFO:tasks.ceph:Archiving crash dumps... 2024-04-23T02:00:21.057 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/lib/ceph/crash to /home/teuthworker/archive/yingxin-2024-04-23_01:23:55-crimson-rados-ci-yingxin-crimson-fix-tri-mutex-4-distro-default-smithi/7669459/remote/smithi090/crash 2024-04-23T02:00:21.058 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-23T02:00:21.103 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/lib/ceph/crash to /home/teuthworker/archive/yingxin-2024-04-23_01:23:55-crimson-rados-ci-yingxin-crimson-fix-tri-mutex-4-distro-default-smithi/7669459/remote/smithi113/crash 2024-04-23T02:00:21.104 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-23T02:00:21.141 INFO:tasks.ceph:Compressing logs... 2024-04-23T02:00:21.141 DEBUG:teuthology.orchestra.run.smithi090:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T02:00:21.148 DEBUG:teuthology.orchestra.run.smithi113:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T02:00:21.215 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26967.log 2024-04-23T02:00:21.215 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-23T02:00:21.215 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-23T02:00:21.215 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.26967.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.26967.log.gz 2024-04-23T02:00:21.215 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-23T02:00:21.216 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26927.log 2024-04-23T02:00:21.216 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26847.log 2024-04-23T02:00:21.216 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.26927.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.26927.log.gz 2024-04-23T02:00:21.216 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26887.log 2024-04-23T02:00:21.216 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.26847.log: /var/log/ceph/ceph-client.admin.26887.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.26847.log.gz 2024-04-23T02:00:21.216 INFO:teuthology.orchestra.run.smithi113.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.26887.log.gz 2024-04-23T02:00:21.332 INFO:teuthology.orchestra.run.smithi113.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-23T02:00:21.550 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57313.log 2024-04-23T02:00:21.550 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62533.log 2024-04-23T02:00:21.550 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31385.log 2024-04-23T02:00:21.550 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.57313.log: 47.6% -- replaced with /var/log/ceph/ceph-client.admin.57313.log.gz 2024-04-23T02:00:21.550 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.62533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27178.log 2024-04-23T02:00:21.551 INFO:teuthology.orchestra.run.smithi090.stderr: 47.6% -- replaced with /var/log/ceph/ceph-client.admin.62533.log.gz 2024-04-23T02:00:21.551 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28520.log 2024-04-23T02:00:21.551 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.31385.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31385.log.gz 2024-04-23T02:00:21.551 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.27178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61155.log 2024-04-23T02:00:21.551 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.27178.log.gz 2024-04-23T02:00:21.551 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.28520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41304.log 2024-04-23T02:00:21.551 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.28520.log.gz 2024-04-23T02:00:21.551 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.61155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48439.log 2024-04-23T02:00:21.551 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.61155.log.gz 2024-04-23T02:00:21.551 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.41304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53827.log 2024-04-23T02:00:21.551 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41304.log.gz 2024-04-23T02:00:21.551 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.48439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53329.log 2024-04-23T02:00:21.552 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48439.log.gz 2024-04-23T02:00:21.552 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.53827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39698.log 2024-04-23T02:00:21.552 INFO:teuthology.orchestra.run.smithi090.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.53827.log.gz 2024-04-23T02:00:21.552 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.53329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48041.log 2024-04-23T02:00:21.552 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53329.log.gz 2024-04-23T02:00:21.552 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.39698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27633.log 2024-04-23T02:00:21.552 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39698.log.gz 2024-04-23T02:00:21.552 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.48041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31773.log 2024-04-23T02:00:21.552 INFO:teuthology.orchestra.run.smithi090.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.48041.log.gz 2024-04-23T02:00:21.553 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.27633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37437.log 2024-04-23T02:00:21.553 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.27633.log.gz 2024-04-23T02:00:21.553 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.31773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36285.log 2024-04-23T02:00:21.553 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.31773.log.gz 2024-04-23T02:00:21.553 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42327.log 2024-04-23T02:00:21.553 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37437.log.gz 2024-04-23T02:00:21.553 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.36285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66384.log 2024-04-23T02:00:21.553 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36285.log.gz 2024-04-23T02:00:21.553 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.42327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63537.log 2024-04-23T02:00:21.553 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42327.log.gz 2024-04-23T02:00:21.554 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.66384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49640.log 2024-04-23T02:00:21.554 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.66384.log.gz 2024-04-23T02:00:21.554 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.63537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62441.log 2024-04-23T02:00:21.554 INFO:teuthology.orchestra.run.smithi090.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.63537.log.gz 2024-04-23T02:00:21.554 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.49640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31902.log 2024-04-23T02:00:21.554 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49640.log.gz 2024-04-23T02:00:21.554 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.62441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38075.log 2024-04-23T02:00:21.554 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.62441.log.gz 2024-04-23T02:00:21.555 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.31902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32807.log 2024-04-23T02:00:21.555 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31902.log.gz 2024-04-23T02:00:21.555 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48697.log 2024-04-23T02:00:21.555 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38075.log.gz 2024-04-23T02:00:21.555 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.32807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41918.log 2024-04-23T02:00:21.555 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.32807.log.gz 2024-04-23T02:00:21.555 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.48697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41366.log 2024-04-23T02:00:21.555 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48697.log.gz 2024-04-23T02:00:21.555 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.41918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32758.log 2024-04-23T02:00:21.555 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41918.log.gz 2024-04-23T02:00:21.555 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.41366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56906.log 2024-04-23T02:00:21.556 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41366.log.gz 2024-04-23T02:00:21.556 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.32758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56459.log 2024-04-23T02:00:21.556 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32758.log.gz 2024-04-23T02:00:21.556 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.56906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48657.log 2024-04-23T02:00:21.556 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.56906.log.gz 2024-04-23T02:00:21.556 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.56459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54454.log 2024-04-23T02:00:21.556 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.56459.log.gz 2024-04-23T02:00:21.556 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.48657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57393.log 2024-04-23T02:00:21.556 INFO:teuthology.orchestra.run.smithi090.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.48657.log.gz 2024-04-23T02:00:21.556 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.54454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57491.log 2024-04-23T02:00:21.556 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.54454.log.gz 2024-04-23T02:00:21.556 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29773.log 2024-04-23T02:00:21.556 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.57393.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.57393.log.gz 2024-04-23T02:00:21.557 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.57491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42047.log 2024-04-23T02:00:21.557 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.57491.log.gz 2024-04-23T02:00:21.557 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.29773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32716.log 2024-04-23T02:00:21.557 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.29773.log.gz 2024-04-23T02:00:21.557 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.42047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40357.log 2024-04-23T02:00:21.557 INFO:teuthology.orchestra.run.smithi090.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.42047.log.gz 2024-04-23T02:00:21.557 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.32716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60395.log 2024-04-23T02:00:21.557 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32716.log.gz 2024-04-23T02:00:21.557 INFO:teuthology.orchestra.run.smithi090.stderr:gzip/var/log/ceph/ceph-client.admin.40357.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.48837.log 2024-04-23T02:00:21.557 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40357.log.gz 2024-04-23T02:00:21.557 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.60395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33341.log 2024-04-23T02:00:21.557 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.60395.log.gz 2024-04-23T02:00:21.557 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.48837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64391.log 2024-04-23T02:00:21.558 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48837.log.gz 2024-04-23T02:00:21.558 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45757.log 2024-04-23T02:00:21.558 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.33341.log.gz 2024-04-23T02:00:21.558 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.64391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62211.log 2024-04-23T02:00:21.558 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.64391.log.gz 2024-04-23T02:00:21.558 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.45757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53707.log 2024-04-23T02:00:21.558 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45757.log.gz 2024-04-23T02:00:21.558 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.62211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33730.log 2024-04-23T02:00:21.558 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.62211.log.gz 2024-04-23T02:00:21.558 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.53707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62959.log 2024-04-23T02:00:21.558 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53707.log.gz 2024-04-23T02:00:21.558 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40588.log 2024-04-23T02:00:21.559 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33730.log.gz 2024-04-23T02:00:21.559 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.62959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44605.log 2024-04-23T02:00:21.559 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.62959.log.gz 2024-04-23T02:00:21.559 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.40588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49989.log 2024-04-23T02:00:21.559 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40588.log.gz 2024-04-23T02:00:21.559 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.44605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49362.log 2024-04-23T02:00:21.559 INFO:teuthology.orchestra.run.smithi090.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.44605.log.gz 2024-04-23T02:00:21.559 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.49989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48935.log 2024-04-23T02:00:21.559 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49989.log.gz 2024-04-23T02:00:21.559 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.49362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62351.log 2024-04-23T02:00:21.559 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49362.log.gz 2024-04-23T02:00:21.559 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.48935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36803.log 2024-04-23T02:00:21.560 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48935.log.gz 2024-04-23T02:00:21.560 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56786.log 2024-04-23T02:00:21.560 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.62351.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.62351.log.gz 2024-04-23T02:00:21.560 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.36803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47734.log 2024-04-23T02:00:21.560 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36803.log.gz 2024-04-23T02:00:21.560 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.56786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30599.log 2024-04-23T02:00:21.560 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.56786.log.gz 2024-04-23T02:00:21.560 INFO:teuthology.orchestra.run.smithi090.stderr:gzip/var/log/ceph/ceph-client.admin.47734.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.27735.log 2024-04-23T02:00:21.560 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47734.log.gz 2024-04-23T02:00:21.560 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.30599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33959.log 2024-04-23T02:00:21.560 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30599.log.gz 2024-04-23T02:00:21.560 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.27735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61638.log 2024-04-23T02:00:21.560 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.27735.log.gz 2024-04-23T02:00:21.561 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42376.log 2024-04-23T02:00:21.561 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.33959.log.gz 2024-04-23T02:00:21.561 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.61638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32509.log 2024-04-23T02:00:21.561 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.61638.log.gz 2024-04-23T02:00:21.561 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.42376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60833.log 2024-04-23T02:00:21.561 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42376.log.gz 2024-04-23T02:00:21.561 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.32509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57044.log 2024-04-23T02:00:21.561 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32509.log.gz 2024-04-23T02:00:21.561 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.60833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27549.log 2024-04-23T02:00:21.561 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.60833.log.gz 2024-04-23T02:00:21.561 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.57044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60218.log 2024-04-23T02:00:21.561 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.57044.log.gz 2024-04-23T02:00:21.561 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.27549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31426.log 2024-04-23T02:00:21.562 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.27549.log.gz 2024-04-23T02:00:21.562 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.60218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48090.log 2024-04-23T02:00:21.562 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.60218.log.gz 2024-04-23T02:00:21.562 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.31426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30906.log 2024-04-23T02:00:21.562 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.31426.log.gz 2024-04-23T02:00:21.562 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.48090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64251.log 2024-04-23T02:00:21.562 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48090.log.gz 2024-04-23T02:00:21.562 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.30906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52813.log 2024-04-23T02:00:21.563 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30906.log.gz 2024-04-23T02:00:21.563 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.64251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31172.log 2024-04-23T02:00:21.563 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.64251.log.gz 2024-04-23T02:00:21.563 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.52813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56597.log 2024-04-23T02:00:21.563 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.52813.log.gz 2024-04-23T02:00:21.563 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.31172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66784.log 2024-04-23T02:00:21.563 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.31172.log.gz 2024-04-23T02:00:21.563 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.56597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35596.log 2024-04-23T02:00:21.563 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56597.log.gz 2024-04-23T02:00:21.564 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.66784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35636.log 2024-04-23T02:00:21.564 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.66784.log.gz 2024-04-23T02:00:21.564 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36663.log 2024-04-23T02:00:21.564 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35596.log.gz 2024-04-23T02:00:21.564 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29971.log 2024-04-23T02:00:21.564 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35636.log.gz 2024-04-23T02:00:21.564 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.36663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58291.log 2024-04-23T02:00:21.564 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36663.log.gz 2024-04-23T02:00:21.565 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.29971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59504.log 2024-04-23T02:00:21.565 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29971.log.gz 2024-04-23T02:00:21.565 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.58291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64111.log 2024-04-23T02:00:21.565 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.58291.log.gz 2024-04-23T02:00:21.565 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.59504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65166.log 2024-04-23T02:00:21.565 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.59504.log.gz 2024-04-23T02:00:21.565 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.64111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34237.log 2024-04-23T02:00:21.565 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.64111.log.gz 2024-04-23T02:00:21.565 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.65166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47405.log 2024-04-23T02:00:21.565 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.65166.log.gz 2024-04-23T02:00:21.565 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58251.log 2024-04-23T02:00:21.566 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34237.log.gz 2024-04-23T02:00:21.566 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.47405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54056.log 2024-04-23T02:00:21.566 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47405.log.gz 2024-04-23T02:00:21.566 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.58251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46174.log 2024-04-23T02:00:21.566 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.58251.log.gz 2024-04-23T02:00:21.566 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.54056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54592.log 2024-04-23T02:00:21.566 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.54056.log.gz 2024-04-23T02:00:21.566 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.0.46174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52933.log 2024-04-23T02:00:21.566 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.54592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36145.log 2024-04-23T02:00:21.567 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54592.log.gz 2024-04-23T02:00:21.567 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.52933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42187.log 2024-04-23T02:00:21.569 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.52933.log.gz 2024-04-23T02:00:21.570 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.36145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49451.log 2024-04-23T02:00:21.570 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36145.log.gz 2024-04-23T02:00:21.570 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.42187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58604.log 2024-04-23T02:00:21.570 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42187.log.gz 2024-04-23T02:00:21.570 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.49451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66216.log 2024-04-23T02:00:21.570 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49451.log.gz 2024-04-23T02:00:21.571 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.58604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51990.log 2024-04-23T02:00:21.571 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.58604.log.gz 2024-04-23T02:00:21.571 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.66216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44156.log 2024-04-23T02:00:21.571 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.66216.log.gz 2024-04-23T02:00:21.571 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.51990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63982.log 2024-04-23T02:00:21.571 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51990.log.gz 2024-04-23T02:00:21.571 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.44156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62750.log 2024-04-23T02:00:21.572 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44156.log.gz 2024-04-23T02:00:21.572 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.63982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45535.log 2024-04-23T02:00:21.572 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.63982.log.gz 2024-04-23T02:00:21.572 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.62750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29571.log 2024-04-23T02:00:21.572 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.62750.log.gz 2024-04-23T02:00:21.572 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.45535.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-23T02:00:21.572 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45535.log.gz 2024-04-23T02:00:21.572 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.29571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60091.log 2024-04-23T02:00:21.572 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29571.log.gz 2024-04-23T02:00:21.573 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43222.log 2024-04-23T02:00:21.573 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.60091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28221.log 2024-04-23T02:00:21.573 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.60091.log.gz 2024-04-23T02:00:21.573 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.43222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41998.log 2024-04-23T02:00:21.573 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43222.log.gz 2024-04-23T02:00:21.573 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.28221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28649.log 2024-04-23T02:00:21.573 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28221.log.gz 2024-04-23T02:00:21.573 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.41998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38026.log 2024-04-23T02:00:21.574 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41998.log.gz 2024-04-23T02:00:21.574 INFO:teuthology.orchestra.run.smithi090.stderr: 87.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-23T02:00:21.574 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.28649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28907.log 2024-04-23T02:00:21.574 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.28649.log.gz 2024-04-23T02:00:21.574 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30728.log 2024-04-23T02:00:21.574 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38026.log.gz 2024-04-23T02:00:21.574 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.28907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58429.log 2024-04-23T02:00:21.574 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28907.log.gz 2024-04-23T02:00:21.575 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27843.log 2024-04-23T02:00:21.575 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.30728.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30728.log.gz 2024-04-23T02:00:21.575 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.58429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38095.log 2024-04-23T02:00:21.575 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.58429.log.gz 2024-04-23T02:00:21.575 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.27843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32489.log 2024-04-23T02:00:21.575 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.27843.log.gz 2024-04-23T02:00:21.575 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58471.log 2024-04-23T02:00:21.575 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38095.log.gz 2024-04-23T02:00:21.576 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.32489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47503.log 2024-04-23T02:00:21.576 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32489.log.gz 2024-04-23T02:00:21.576 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.58471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44374.log 2024-04-23T02:00:21.576 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.58471.log.gz 2024-04-23T02:00:21.576 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.47503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63057.log 2024-04-23T02:00:21.576 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47503.log.gz 2024-04-23T02:00:21.576 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.44374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52348.log 2024-04-23T02:00:21.576 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44374.log.gz 2024-04-23T02:00:21.577 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.63057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63137.log 2024-04-23T02:00:21.577 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.63057.log.gz 2024-04-23T02:00:21.577 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.52348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59903.log 2024-04-23T02:00:21.577 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.52348.log.gz 2024-04-23T02:00:21.577 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.63137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45359.log 2024-04-23T02:00:21.577 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.63137.log.gz 2024-04-23T02:00:21.577 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.59903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55892.log 2024-04-23T02:00:21.577 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.59903.log.gz 2024-04-23T02:00:21.577 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.45359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62093.log 2024-04-23T02:00:21.578 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45359.log.gz 2024-04-23T02:00:21.578 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.55892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29438.log 2024-04-23T02:00:21.578 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55892.log.gz 2024-04-23T02:00:21.578 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.62093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37935.log 2024-04-23T02:00:21.578 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.62093.log.gz 2024-04-23T02:00:21.578 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.29438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49969.log 2024-04-23T02:00:21.578 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.29438.log.gz 2024-04-23T02:00:21.579 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40448.log 2024-04-23T02:00:21.579 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37935.log.gz 2024-04-23T02:00:21.579 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53240.log 2024-04-23T02:00:21.579 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.49969.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49969.log.gz 2024-04-23T02:00:21.579 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.40448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63762.log 2024-04-23T02:00:21.579 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40448.log.gz 2024-04-23T02:00:21.579 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.53240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31080.log 2024-04-23T02:00:21.579 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53240.log.gz 2024-04-23T02:00:21.579 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.63762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55766.log 2024-04-23T02:00:21.580 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.63762.log.gz 2024-04-23T02:00:21.580 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.31080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32360.log 2024-04-23T02:00:21.580 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31080.log.gz 2024-04-23T02:00:21.580 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.55766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35229.log 2024-04-23T02:00:21.580 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55766.log.gz 2024-04-23T02:00:21.580 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.32360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52675.log 2024-04-23T02:00:21.580 INFO:teuthology.orchestra.run.smithi090.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.32360.log.gz 2024-04-23T02:00:21.580 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32409.log 2024-04-23T02:00:21.581 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35229.log.gz 2024-04-23T02:00:21.581 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.52675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65734.log 2024-04-23T02:00:21.581 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.52675.log.gz 2024-04-23T02:00:21.581 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.32409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60154.log 2024-04-23T02:00:21.581 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.32409.log.gz 2024-04-23T02:00:21.581 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.65734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51004.log 2024-04-23T02:00:21.581 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.65734.log.gz 2024-04-23T02:00:21.581 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.60154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47007.log 2024-04-23T02:00:21.581 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.60154.log.gz 2024-04-23T02:00:21.581 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.51004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35774.log 2024-04-23T02:00:21.582 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51004.log.gz 2024-04-23T02:00:21.582 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.47007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40308.log 2024-04-23T02:00:21.582 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47007.log.gz 2024-04-23T02:00:21.582 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38522.log 2024-04-23T02:00:21.582 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35774.log.gz 2024-04-23T02:00:21.582 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.40308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27457.log 2024-04-23T02:00:21.582 INFO:teuthology.orchestra.run.smithi090.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.40308.log.gz 2024-04-23T02:00:21.582 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29911.log 2024-04-23T02:00:21.583 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38522.log.gz 2024-04-23T02:00:21.583 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.27457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30947.log 2024-04-23T02:00:21.583 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.27457.log.gz 2024-04-23T02:00:21.583 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.29911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63475.log 2024-04-23T02:00:21.583 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.29911.log.gz 2024-04-23T02:00:21.583 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.30947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60290.log 2024-04-23T02:00:21.583 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30947.log.gz 2024-04-23T02:00:21.583 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.63475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61953.log 2024-04-23T02:00:21.583 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.63475.log.gz 2024-04-23T02:00:21.583 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.60290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36992.log 2024-04-23T02:00:21.583 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.60290.log.gz 2024-04-23T02:00:21.584 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.61953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45666.log 2024-04-23T02:00:21.584 INFO:teuthology.orchestra.run.smithi090.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.61953.log.gz 2024-04-23T02:00:21.584 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.36992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50151.log 2024-04-23T02:00:21.584 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36992.log.gz 2024-04-23T02:00:21.584 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.45666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49282.log 2024-04-23T02:00:21.584 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45666.log.gz 2024-04-23T02:00:21.584 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.50151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53071.log 2024-04-23T02:00:21.584 INFO:teuthology.orchestra.run.smithi090.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.50151.log.gz 2024-04-23T02:00:21.584 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.49282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31257.log 2024-04-23T02:00:21.585 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49282.log.gz 2024-04-23T02:00:21.585 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.53071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42871.log 2024-04-23T02:00:21.585 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53071.log.gz 2024-04-23T02:00:21.585 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.31257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60567.log 2024-04-23T02:00:21.585 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.31257.log.gz 2024-04-23T02:00:21.585 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.42871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65650.log 2024-04-23T02:00:21.585 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42871.log.gz 2024-04-23T02:00:21.585 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.60567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29613.log 2024-04-23T02:00:21.585 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.60567.log.gz 2024-04-23T02:00:21.586 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.65650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40677.log 2024-04-23T02:00:21.586 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.65650.log.gz 2024-04-23T02:00:21.586 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.29613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58939.log 2024-04-23T02:00:21.586 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29613.log.gz 2024-04-23T02:00:21.586 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.40677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44863.log 2024-04-23T02:00:21.586 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40677.log.gz 2024-04-23T02:00:21.586 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.58939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55246.log 2024-04-23T02:00:21.586 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.58939.log.gz 2024-04-23T02:00:21.586 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.44863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66088.log 2024-04-23T02:00:21.587 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44863.log.gz 2024-04-23T02:00:21.587 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.55246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39418.log 2024-04-23T02:00:21.587 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.55246.log.gz 2024-04-23T02:00:21.587 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.66088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40744.log 2024-04-23T02:00:21.587 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.66088.log.gz 2024-04-23T02:00:21.587 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.39418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32856.log 2024-04-23T02:00:21.587 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39418.log.gz 2024-04-23T02:00:21.587 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.40744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50570.log 2024-04-23T02:00:21.587 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40744.log.gz 2024-04-23T02:00:21.588 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.32856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61679.log 2024-04-23T02:00:21.588 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32856.log.gz 2024-04-23T02:00:21.588 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.50570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50089.log 2024-04-23T02:00:21.588 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50570.log.gz 2024-04-23T02:00:21.588 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.61679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43405.log 2024-04-23T02:00:21.588 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.61679.log.gz 2024-04-23T02:00:21.588 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.50089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36570.log 2024-04-23T02:00:21.588 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50089.log.gz 2024-04-23T02:00:21.588 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.43405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36892.log 2024-04-23T02:00:21.588 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43405.log.gz 2024-04-23T02:00:21.589 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.36570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58073.log 2024-04-23T02:00:21.589 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36570.log.gz 2024-04-23T02:00:21.589 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.36892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50906.log 2024-04-23T02:00:21.589 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36892.log.gz 2024-04-23T02:00:21.589 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.58073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31724.log 2024-04-23T02:00:21.589 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.58073.log.gz 2024-04-23T02:00:21.589 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.50906.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-23T02:00:21.589 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50906.log.gz 2024-04-23T02:00:21.589 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.31724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33290.log 2024-04-23T02:00:21.590 INFO:teuthology.orchestra.run.smithi090.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.31724.log.gz 2024-04-23T02:00:21.590 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34664.log 2024-04-23T02:00:21.590 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51892.log 2024-04-23T02:00:21.590 INFO:teuthology.orchestra.run.smithi090.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.33290.log.gz 2024-04-23T02:00:21.590 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35278.log 2024-04-23T02:00:21.590 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34664.log.gz 2024-04-23T02:00:21.590 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.51892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41433.log 2024-04-23T02:00:21.590 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51892.log.gz 2024-04-23T02:00:21.591 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35278.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.25858.log 2024-04-23T02:00:21.591 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35278.log.gz 2024-04-23T02:00:21.591 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.41433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39460.log 2024-04-23T02:00:21.591 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41433.log.gz 2024-04-23T02:00:21.591 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph.tmp-client.admin.25858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31466.log 2024-04-23T02:00:21.591 INFO:teuthology.orchestra.run.smithi090.stderr: 88.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-23T02:00:21.591 INFO:teuthology.orchestra.run.smithi090.stderr: 67.2% -- replaced with /var/log/ceph/ceph.tmp-client.admin.25858.log.gz 2024-04-23T02:00:21.591 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.39460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67120.log 2024-04-23T02:00:21.591 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39460.log.gz 2024-04-23T02:00:21.592 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.31466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37675.log 2024-04-23T02:00:21.592 INFO:teuthology.orchestra.run.smithi090.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.31466.log.gz 2024-04-23T02:00:21.592 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.67120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56379.log 2024-04-23T02:00:21.592 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.67120.log.gz 2024-04-23T02:00:21.592 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61195.log 2024-04-23T02:00:21.592 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37675.log.gz 2024-04-23T02:00:21.592 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.56379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64657.log 2024-04-23T02:00:21.592 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.56379.log.gz 2024-04-23T02:00:21.592 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.61195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37977.log 2024-04-23T02:00:21.593 INFO:teuthology.orchestra.run.smithi090.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.61195.log.gz 2024-04-23T02:00:21.593 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.64657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66994.log 2024-04-23T02:00:21.593 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.64657.log.gz 2024-04-23T02:00:21.593 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36932.log 2024-04-23T02:00:21.593 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37977.log.gz 2024-04-23T02:00:21.593 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.66994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47276.log 2024-04-23T02:00:21.593 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.66994.log.gz 2024-04-23T02:00:21.593 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.36932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52953.log 2024-04-23T02:00:21.593 INFO:teuthology.orchestra.run.smithi090.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.36932.log.gz 2024-04-23T02:00:21.593 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.47276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31564.log 2024-04-23T02:00:21.594 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47276.log.gz 2024-04-23T02:00:21.594 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.52953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65917.log 2024-04-23T02:00:21.594 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.52953.log.gz 2024-04-23T02:00:21.594 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.31564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59295.log 2024-04-23T02:00:21.594 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31564.log.gz 2024-04-23T02:00:21.594 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.65917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57582.log 2024-04-23T02:00:21.594 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.65917.log.gz 2024-04-23T02:00:21.594 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.59295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61057.log 2024-04-23T02:00:21.594 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.59295.log.gz 2024-04-23T02:00:21.594 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.57582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61106.log 2024-04-23T02:00:21.594 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.57582.log.gz 2024-04-23T02:00:21.595 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.61057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50661.log 2024-04-23T02:00:21.595 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.61057.log.gz 2024-04-23T02:00:21.595 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.61106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40697.log 2024-04-23T02:00:21.595 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.61106.log.gz 2024-04-23T02:00:21.595 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.50661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28179.log 2024-04-23T02:00:21.595 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50661.log.gz 2024-04-23T02:00:21.595 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.40697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33610.log 2024-04-23T02:00:21.595 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40697.log.gz 2024-04-23T02:00:21.595 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.28179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52030.log 2024-04-23T02:00:21.595 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28179.log.gz 2024-04-23T02:00:21.595 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64699.log 2024-04-23T02:00:21.596 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33610.log.gz 2024-04-23T02:00:21.596 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.52030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50346.log 2024-04-23T02:00:21.596 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.52030.log.gz 2024-04-23T02:00:21.596 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.64699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49871.log 2024-04-23T02:00:21.596 INFO:teuthology.orchestra.run.smithi090.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.64699.log.gz 2024-04-23T02:00:21.596 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.50346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47685.log 2024-04-23T02:00:21.596 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50346.log.gz 2024-04-23T02:00:21.596 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.49871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54712.log 2024-04-23T02:00:21.596 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49871.log.gz 2024-04-23T02:00:21.596 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.47685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35418.log 2024-04-23T02:00:21.596 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47685.log.gz 2024-04-23T02:00:21.597 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.54712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43978.log 2024-04-23T02:00:21.597 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54712.log.gz 2024-04-23T02:00:21.597 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34148.log 2024-04-23T02:00:21.597 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35418.log.gz 2024-04-23T02:00:21.597 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.43978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30179.log 2024-04-23T02:00:21.597 INFO:teuthology.orchestra.run.smithi090.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.43978.log.gz 2024-04-23T02:00:21.597 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.28592.log 2024-04-23T02:00:21.597 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34148.log.gz 2024-04-23T02:00:21.597 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.30179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28738.log 2024-04-23T02:00:21.597 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30179.log.gz 2024-04-23T02:00:21.597 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.0.28592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40168.log 2024-04-23T02:00:21.597 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28858.log 2024-04-23T02:00:21.598 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.28738.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.28738.log.gz 2024-04-23T02:00:21.598 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.40168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43809.log 2024-04-23T02:00:21.598 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40168.log.gz 2024-04-23T02:00:21.598 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34197.log 2024-04-23T02:00:21.598 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.28858.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.28858.log.gz 2024-04-23T02:00:21.598 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.43809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59424.log 2024-04-23T02:00:21.598 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43809.log.gz 2024-04-23T02:00:21.598 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28778.log 2024-04-23T02:00:21.598 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34197.log.gz 2024-04-23T02:00:21.598 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.59424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46509.log 2024-04-23T02:00:21.598 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.59424.log.gz 2024-04-23T02:00:21.599 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.28778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44236.log 2024-04-23T02:00:21.599 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.28778.log.gz 2024-04-23T02:00:21.599 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.46509.log: 49.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.46509.log.gz 2024-04-23T02:00:21.599 INFO:teuthology.orchestra.run.smithi090.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.60525.log 2024-04-23T02:00:21.599 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.44236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56995.log 2024-04-23T02:00:21.599 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44236.log.gz 2024-04-23T02:00:21.599 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.60525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33868.log 2024-04-23T02:00:21.599 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.60525.log.gz 2024-04-23T02:00:21.599 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.56995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30857.log 2024-04-23T02:00:21.599 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56995.log.gz 2024-04-23T02:00:21.599 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53558.log 2024-04-23T02:00:21.599 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33868.log.gz 2024-04-23T02:00:21.600 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.30857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40944.log 2024-04-23T02:00:21.600 INFO:teuthology.orchestra.run.smithi090.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.30857.log.gz 2024-04-23T02:00:21.600 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.53558.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.53558.log.gz 2024-04-23T02:00:21.600 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53420.log 2024-04-23T02:00:21.600 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.40944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42718.log 2024-04-23T02:00:21.600 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40944.log.gz 2024-04-23T02:00:21.600 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.53420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63315.log 2024-04-23T02:00:21.600 INFO:teuthology.orchestra.run.smithi090.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.53420.log.gz 2024-04-23T02:00:21.600 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.42718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55633.log 2024-04-23T02:00:21.600 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42718.log.gz 2024-04-23T02:00:21.600 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.63315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46420.log 2024-04-23T02:00:21.601 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.63315.log.gz 2024-04-23T02:00:21.601 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.55633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29653.log 2024-04-23T02:00:21.601 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.55633.log.gz 2024-04-23T02:00:21.601 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.46420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46867.log 2024-04-23T02:00:21.601 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46420.log.gz 2024-04-23T02:00:21.601 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.29653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50216.log 2024-04-23T02:00:21.601 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29653.log.gz 2024-04-23T02:00:21.601 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.46867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45489.log 2024-04-23T02:00:21.601 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46867.log.gz 2024-04-23T02:00:21.601 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.50216.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50216.log.gz 2024-04-23T02:00:21.602 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56826.log 2024-04-23T02:00:21.602 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.45489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66364.log 2024-04-23T02:00:21.602 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45489.log.gz 2024-04-23T02:00:21.602 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.56826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41611.log 50.0% 2024-04-23T02:00:21.602 INFO:teuthology.orchestra.run.smithi090.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56826.log.gz 2024-04-23T02:00:21.602 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.66364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42565.log 2024-04-23T02:00:21.602 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.66364.log.gz 2024-04-23T02:00:21.602 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.41611.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41611.log.gz 2024-04-23T02:00:21.602 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51136.log 2024-04-23T02:00:21.602 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.42565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39227.log 2024-04-23T02:00:21.602 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42565.log.gz 2024-04-23T02:00:21.603 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.51136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63177.log 2024-04-23T02:00:21.603 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51136.log.gz 2024-04-23T02:00:21.603 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.39227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63266.log 2024-04-23T02:00:21.603 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39227.log.gz 2024-04-23T02:00:21.603 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.63177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45150.log 2024-04-23T02:00:21.603 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.63177.log.gz 2024-04-23T02:00:21.603 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.63266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49680.log 2024-04-23T02:00:21.603 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.63266.log.gz 2024-04-23T02:00:21.603 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.45150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42118.log 2024-04-23T02:00:21.603 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45150.log.gz 2024-04-23T02:00:21.603 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.49680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50746.log 2024-04-23T02:00:21.603 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49680.log.gz 2024-04-23T02:00:21.604 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.42118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42893.log 2024-04-23T02:00:21.604 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42118.log.gz 2024-04-23T02:00:21.604 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.50746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47454.log 2024-04-23T02:00:21.604 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50746.log.gz 2024-04-23T02:00:21.604 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.42893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54105.log 2024-04-23T02:00:21.604 INFO:teuthology.orchestra.run.smithi090.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.42893.log.gz 2024-04-23T02:00:21.604 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.47454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67247.log 2024-04-23T02:00:21.604 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47454.log.gz 2024-04-23T02:00:21.604 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.54105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40497.log 2024-04-23T02:00:21.604 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.54105.log.gz 2024-04-23T02:00:21.604 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.67247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41264.log 2024-04-23T02:00:21.604 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.67247.log.gz 2024-04-23T02:00:21.605 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.40497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51809.log 2024-04-23T02:00:21.605 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40497.log.gz 2024-04-23T02:00:21.605 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.41264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39978.log 2024-04-23T02:00:21.605 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41264.log.gz 2024-04-23T02:00:21.605 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.51809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46063.log 2024-04-23T02:00:21.605 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51809.log.gz 2024-04-23T02:00:21.605 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.39978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40119.log 2024-04-23T02:00:21.605 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39978.log.gz 2024-04-23T02:00:21.605 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.46063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51634.log 2024-04-23T02:00:21.605 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46063.log.gz 2024-04-23T02:00:21.605 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.40119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27150.log 2024-04-23T02:00:21.605 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40119.log.gz 2024-04-23T02:00:21.605 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.51634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61855.log 2024-04-23T02:00:21.606 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51634.log.gz 2024-04-23T02:00:21.606 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.27150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47196.log 2024-04-23T02:00:21.606 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.27150.log.gz 2024-04-23T02:00:21.606 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.61855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66086.log 2024-04-23T02:00:21.606 INFO:teuthology.orchestra.run.smithi090.stderr: 48.0% -- replaced with /var/log/ceph/ceph-client.admin.61855.log.gz 2024-04-23T02:00:21.606 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.47196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52555.log 2024-04-23T02:00:21.606 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47196.log.gz 2024-04-23T02:00:21.606 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.66086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40637.log 2024-04-23T02:00:21.606 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.66086.log.gz 2024-04-23T02:00:21.606 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.52555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53518.log 2024-04-23T02:00:21.606 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.52555.log.gz 2024-04-23T02:00:21.606 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.40637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38662.log 2024-04-23T02:00:21.607 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40637.log.gz 2024-04-23T02:00:21.607 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.53518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39789.log 2024-04-23T02:00:21.607 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.53518.log.gz 2024-04-23T02:00:21.607 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55324.log 2024-04-23T02:00:21.607 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38662.log.gz 2024-04-23T02:00:21.607 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.39789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57631.log 2024-04-23T02:00:21.607 INFO:teuthology.orchestra.run.smithi090.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.39789.log.gz 2024-04-23T02:00:21.607 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.55324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45310.log 2024-04-23T02:00:21.607 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.55324.log.gz 2024-04-23T02:00:21.607 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.57631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27903.log 2024-04-23T02:00:21.607 INFO:teuthology.orchestra.run.smithi090.stderr: 48.0% -- replaced with /var/log/ceph/ceph-client.admin.57631.log.gz 2024-04-23T02:00:21.608 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.45310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27734.log 2024-04-23T02:00:21.608 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45310.log.gz 2024-04-23T02:00:21.608 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.27903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44067.log 2024-04-23T02:00:21.608 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.27903.log.gz 2024-04-23T02:00:21.608 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.27734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32618.log 2024-04-23T02:00:21.608 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.27734.log.gz 2024-04-23T02:00:21.608 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.44067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48795.log 2024-04-23T02:00:21.608 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44067.log.gz 2024-04-23T02:00:21.608 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.32618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47098.log 2024-04-23T02:00:21.608 INFO:teuthology.orchestra.run.smithi090.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.32618.log.gz 2024-04-23T02:00:21.608 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.48795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64819.log 2024-04-23T02:00:21.608 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48795.log.gz 2024-04-23T02:00:21.609 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.47098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40069.log 2024-04-23T02:00:21.609 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47098.log.gz 2024-04-23T02:00:21.609 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.64819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34902.log 2024-04-23T02:00:21.609 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.64819.log.gz 2024-04-23T02:00:21.609 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.40069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67036.log 2024-04-23T02:00:21.609 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40069.log.gz 2024-04-23T02:00:21.609 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55404.log 2024-04-23T02:00:21.609 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34902.log.gz 2024-04-23T02:00:21.609 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.67036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51585.log 2024-04-23T02:00:21.609 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.67036.log.gz 2024-04-23T02:00:21.609 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.55404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64908.log 2024-04-23T02:00:21.609 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55404.log.gz 2024-04-23T02:00:21.610 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.51585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46727.log 2024-04-23T02:00:21.610 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51585.log.gz 2024-04-23T02:00:21.610 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.64908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46469.log 2024-04-23T02:00:21.610 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.64908.log.gz 2024-04-23T02:00:21.610 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.46727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34297.log 2024-04-23T02:00:21.610 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46727.log.gz 2024-04-23T02:00:21.610 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.46469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41740.log 2024-04-23T02:00:21.610 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46469.log.gz 2024-04-23T02:00:21.610 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57982.log 2024-04-23T02:00:21.610 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34297.log.gz 2024-04-23T02:00:21.610 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.41740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49542.log 2024-04-23T02:00:21.610 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41740.log.gz 2024-04-23T02:00:21.611 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.57982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49193.log 2024-04-23T02:00:21.611 INFO:teuthology.orchestra.run.smithi090.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.57982.log.gz 2024-04-23T02:00:21.611 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.49542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52248.log 2024-04-23T02:00:21.611 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49542.log.gz 2024-04-23T02:00:21.611 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.49193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34544.log 2024-04-23T02:00:21.611 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49193.log.gz 2024-04-23T02:00:21.611 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.52248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53867.log 2024-04-23T02:00:21.611 INFO:teuthology.orchestra.run.smithi090.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.52248.log.gz 2024-04-23T02:00:21.611 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39649.log 2024-04-23T02:00:21.611 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34544.log.gz 2024-04-23T02:00:21.611 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.53867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65215.log 2024-04-23T02:00:21.611 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.53867.log.gz 2024-04-23T02:00:21.612 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.39649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44843.log 2024-04-23T02:00:21.612 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39649.log.gz 2024-04-23T02:00:21.612 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.65215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44703.log 2024-04-23T02:00:21.612 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.65215.log.gz 2024-04-23T02:00:21.612 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.44843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47356.log 2024-04-23T02:00:21.612 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44843.log.gz 2024-04-23T02:00:21.612 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.44703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65692.log 2024-04-23T02:00:21.612 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44703.log.gz 2024-04-23T02:00:21.612 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.47356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42698.log 2024-04-23T02:00:21.612 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47356.log.gz 2024-04-23T02:00:21.612 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.65692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46329.log 2024-04-23T02:00:21.612 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.65692.log.gz 2024-04-23T02:00:21.612 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.42698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30768.log 2024-04-23T02:00:21.613 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42698.log.gz 2024-04-23T02:00:21.613 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.46329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35369.log 2024-04-23T02:00:21.613 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46329.log.gz 2024-04-23T02:00:21.613 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.30768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27401.log 2024-04-23T02:00:21.613 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30768.log.gz 2024-04-23T02:00:21.613 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51941.log 2024-04-23T02:00:21.613 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35369.log.gz 2024-04-23T02:00:21.613 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.27401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61771.log 2024-04-23T02:00:21.613 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.27401.log.gz 2024-04-23T02:00:21.613 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.51941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37130.log 2024-04-23T02:00:21.613 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51941.log.gz 2024-04-23T02:00:21.613 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.61771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59861.log 2024-04-23T02:00:21.614 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.61771.log.gz 2024-04-23T02:00:21.614 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46549.log 2024-04-23T02:00:21.614 INFO:teuthology.orchestra.run.smithi090.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.37130.log.gz 2024-04-23T02:00:21.614 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.59861.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-23T02:00:21.614 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.59861.log.gz 2024-04-23T02:00:21.614 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.46549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57722.log 2024-04-23T02:00:21.614 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46549.log.gz 2024-04-23T02:00:21.614 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43669.log 2024-04-23T02:00:21.614 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.57722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64977.log 2024-04-23T02:00:21.614 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.57722.log.gz 2024-04-23T02:00:21.614 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53022.log 2024-04-23T02:00:21.615 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.43669.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43669.log.gz 2024-04-23T02:00:21.615 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.64977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58024.log 2024-04-23T02:00:21.615 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.64977.log.gz 2024-04-23T02:00:21.615 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.53022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26914.log 2024-04-23T02:00:21.615 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53022.log.gz 2024-04-23T02:00:21.615 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57826.log 2024-04-23T02:00:21.615 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.58024.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.58024.log.gz 2024-04-23T02:00:21.615 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.26914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63902.log 2024-04-23T02:00:21.615 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.26914.log.gz 2024-04-23T02:00:21.615 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.57826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45270.log 2024-04-23T02:00:21.615 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.57826.log.gz 2024-04-23T02:00:21.616 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30499.log 2024-04-23T02:00:21.616 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.63902.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.63902.log.gz 2024-04-23T02:00:21.616 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.45270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66028.log 2024-04-23T02:00:21.616 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45270.log.gz 2024-04-23T02:00:21.616 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.30499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53787.log 2024-04-23T02:00:21.616 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.30499.log.gz 2024-04-23T02:00:21.616 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57940.log 2024-04-23T02:00:21.616 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.66028.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.66028.log.gz 2024-04-23T02:00:21.616 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.53787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42096.log 2024-04-23T02:00:21.616 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53787.log.gz 2024-04-23T02:00:21.616 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.57940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27733.log 2024-04-23T02:00:21.616 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.57940.log.gz 2024-04-23T02:00:21.617 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45052.log 2024-04-23T02:00:21.617 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.42096.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42096.log.gz 2024-04-23T02:00:21.617 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.27733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59682.log 2024-04-23T02:00:21.617 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.27733.log.gz 2024-04-23T02:00:21.617 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.45052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31862.log 2024-04-23T02:00:21.617 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45052.log.gz 2024-04-23T02:00:21.617 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58331.log 2024-04-23T02:00:21.617 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.59682.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.59682.log.gz 2024-04-23T02:00:21.617 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.31862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38424.log 2024-04-23T02:00:21.617 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31862.log.gz 2024-04-23T02:00:21.617 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.58331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37061.log 2024-04-23T02:00:21.617 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.58331.log.gz 2024-04-23T02:00:21.618 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60354.log 2024-04-23T02:00:21.618 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38424.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38424.log.gz 2024-04-23T02:00:21.618 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52150.log 2024-04-23T02:00:21.618 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37061.log.gz 2024-04-23T02:00:21.618 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.60354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30221.log 2024-04-23T02:00:21.618 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.60354.log.gz 2024-04-23T02:00:21.618 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31604.log/var/log/ceph/ceph-client.admin.52150.log: 2024-04-23T02:00:21.618 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.52150.log.gz 2024-04-23T02:00:21.618 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.30221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46958.log 2024-04-23T02:00:21.618 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.30221.log.gz 2024-04-23T02:00:21.618 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.31604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53120.log 2024-04-23T02:00:21.618 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31604.log.gz 2024-04-23T02:00:21.619 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65397.log 2024-04-23T02:00:21.619 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.46958.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46958.log.gz 2024-04-23T02:00:21.619 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.53120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39738.log 2024-04-23T02:00:21.619 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53120.log.gz 2024-04-23T02:00:21.619 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.65397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37374.log 2024-04-23T02:00:21.619 INFO:teuthology.orchestra.run.smithi090.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.65397.log.gz 2024-04-23T02:00:21.619 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58165.log 2024-04-23T02:00:21.619 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.39738.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.39738.log.gz 2024-04-23T02:00:21.619 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62260.log 2024-04-23T02:00:21.619 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37374.log.gz 2024-04-23T02:00:21.619 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.58165.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.58165.log.gz 2024-04-23T02:00:21.619 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-23T02:00:21.620 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35956.log 2024-04-23T02:00:21.620 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.62260.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.62260.log.gz 2024-04-23T02:00:21.620 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53280.log 2024-04-23T02:00:21.620 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30319.log 2024-04-23T02:00:21.620 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35956.log.gz 2024-04-23T02:00:21.620 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.53280.log: gzip -5 --verbose -- 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53280.log.gz 2024-04-23T02:00:21.620 INFO:teuthology.orchestra.run.smithi090.stderr: /var/log/ceph/ceph-client.admin.63584.log 2024-04-23T02:00:21.620 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.30319.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30319.log.gz 2024-04-23T02:00:21.620 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46023.log 2024-04-23T02:00:21.620 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.63584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47783.log 49.2% -- replaced with /var/log/ceph/ceph-client.admin.63584.log.gz 2024-04-23T02:00:21.620 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-04-23T02:00:21.621 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.46023.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.46023.log.gz 2024-04-23T02:00:21.621 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55019.log 2024-04-23T02:00:21.621 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.47783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43919.log 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47783.log.gz 2024-04-23T02:00:21.621 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-04-23T02:00:21.621 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.55019.log: 49.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62400.log 2024-04-23T02:00:21.621 INFO:teuthology.orchestra.run.smithi090.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55019.log.gz 2024-04-23T02:00:21.621 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.43919.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43919.log.gz 2024-04-23T02:00:21.621 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53200.log 2024-04-23T02:00:21.621 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.62400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36754.log 49.6% -- replaced with /var/log/ceph/ceph-client.admin.62400.log.gz 2024-04-23T02:00:21.621 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-04-23T02:00:21.621 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.53200.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53200.log.gz 2024-04-23T02:00:21.621 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49780.log 2024-04-23T02:00:21.622 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.36754.log: gzip 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36754.log.gz 2024-04-23T02:00:21.622 INFO:teuthology.orchestra.run.smithi090.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46629.log 2024-04-23T02:00:21.622 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.49780.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.49780.log.gz 2024-04-23T02:00:21.622 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50866.log 2024-04-23T02:00:21.622 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.46629.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46629.log.gz 2024-04-23T02:00:21.622 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51852.log 2024-04-23T02:00:21.622 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.50866.log: gzip -5 49.6% --verbose -- /var/log/ceph/ceph-client.admin.45012.log 2024-04-23T02:00:21.622 INFO:teuthology.orchestra.run.smithi090.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50866.log.gz 2024-04-23T02:00:21.622 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.51852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54414.log 2024-04-23T02:00:21.622 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51852.log.gz 2024-04-23T02:00:21.622 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.45012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50610.log 2024-04-23T02:00:21.623 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45012.log.gz 2024-04-23T02:00:21.623 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.54414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41344.log 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54414.log.gz 2024-04-23T02:00:21.623 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-04-23T02:00:21.623 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.50610.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50610.log.gz 2024-04-23T02:00:21.623 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44514.log 2024-04-23T02:00:21.623 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.41344.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41344.log.gz 2024-04-23T02:00:21.623 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54334.log 2024-04-23T02:00:21.623 INFO:teuthology.orchestra.run.smithi090.stderr:gzip/var/log/ceph/ceph-client.admin.44514.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.37201.log 2024-04-23T02:00:21.623 INFO:teuthology.orchestra.run.smithi090.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.44514.log.gz 2024-04-23T02:00:21.623 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.54334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28095.log 2024-04-23T02:00:21.623 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54334.log.gz 2024-04-23T02:00:21.623 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47316.log 2024-04-23T02:00:21.624 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37201.log.gz 2024-04-23T02:00:21.624 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.28095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66910.log 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28095.log.gz 2024-04-23T02:00:21.624 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-04-23T02:00:21.624 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.47316.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47316.log.gz 2024-04-23T02:00:21.624 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56121.log 2024-04-23T02:00:21.624 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.66910.log: gzip -5 --verbose -- 50.0% -- replaced with /var/log/ceph/ceph-client.admin.66910.log.gz 2024-04-23T02:00:21.624 INFO:teuthology.orchestra.run.smithi090.stderr: /var/log/ceph/ceph-client.admin.33383.log 2024-04-23T02:00:21.624 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.56121.log: 49.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61993.log 2024-04-23T02:00:21.624 INFO:teuthology.orchestra.run.smithi090.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56121.log.gz 2024-04-23T02:00:21.624 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65524.log 2024-04-23T02:00:21.624 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33383.log.gz 2024-04-23T02:00:21.624 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.61993.log: gzip -5 --verbose -- 50.0% -- replaced with /var/log/ceph/ceph-client.admin.61993.log.gz 2024-04-23T02:00:21.624 INFO:teuthology.orchestra.run.smithi090.stderr: /var/log/ceph/ceph-client.admin.57233.log 2024-04-23T02:00:21.625 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.65524.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.65524.log.gz 2024-04-23T02:00:21.625 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58988.log 2024-04-23T02:00:21.625 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.57233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29693.log 2024-04-23T02:00:21.625 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.57233.log.gz 2024-04-23T02:00:21.625 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.58988.log: gzip -5 --verbose -- 50.0% -- replaced with /var/log/ceph/ceph-client.admin.58988.log.gz 2024-04-23T02:00:21.625 INFO:teuthology.orchestra.run.smithi090.stderr: /var/log/ceph/ceph-client.admin.32936.log 2024-04-23T02:00:21.625 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.29693.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29693.log.gz 2024-04-23T02:00:21.625 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45617.log 2024-04-23T02:00:21.625 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.32936.log: gzip -5 --verbose -- 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32936.log.gz 2024-04-23T02:00:21.625 INFO:teuthology.orchestra.run.smithi090.stderr: /var/log/ceph/ceph-client.admin.52070.log 2024-04-23T02:00:21.625 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.45617.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45617.log.gz 2024-04-23T02:00:21.625 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55675.log 2024-04-23T02:00:21.626 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.52070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28325.log 2024-04-23T02:00:21.626 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.52070.log.gz 2024-04-23T02:00:21.626 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.55675.log: gzip -5 --verbose -- 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55675.log.gz 2024-04-23T02:00:21.626 INFO:teuthology.orchestra.run.smithi090.stderr: /var/log/ceph/ceph-client.admin.55716.log 2024-04-23T02:00:21.626 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.28325.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28325.log.gz 2024-04-23T02:00:21.626 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38564.log 2024-04-23T02:00:21.626 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.55716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34862.log 2024-04-23T02:00:21.626 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.55716.log.gz 2024-04-23T02:00:21.626 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43711.log 2024-04-23T02:00:21.626 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38564.log.gz 2024-04-23T02:00:21.626 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42585.log 2024-04-23T02:00:21.627 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34862.log.gz 2024-04-23T02:00:21.627 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.43711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47056.log 49.2% -- replaced with /var/log/ceph/ceph-client.admin.43711.log.gz 2024-04-23T02:00:21.627 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-04-23T02:00:21.627 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.42585.log: gzip 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42585.log.gz 2024-04-23T02:00:21.627 INFO:teuthology.orchestra.run.smithi090.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52457.log 2024-04-23T02:00:21.627 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.47056.log: gzip 50.4% -- replaced with /var/log/ceph/ceph-client.admin.47056.log.gz 2024-04-23T02:00:21.627 INFO:teuthology.orchestra.run.smithi090.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.38613.log 2024-04-23T02:00:21.627 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.52457.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.52457.log.gz 2024-04-23T02:00:21.627 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66532.log 2024-04-23T02:00:21.627 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29056.log 2024-04-23T02:00:21.627 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38613.log.gz 2024-04-23T02:00:21.627 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.66532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60875.log 2024-04-23T02:00:21.627 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.66532.log.gz 2024-04-23T02:00:21.628 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59742.log 2024-04-23T02:00:21.628 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.29056.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29056.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56946.log 2024-04-23T02:00:21.628 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-04-23T02:00:21.628 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.60875.log: /var/log/ceph/ceph-client.admin.59742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66260.log 2024-04-23T02:00:21.628 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% 50.0% -- replaced with /var/log/ceph/ceph-client.admin.59742.log.gz 2024-04-23T02:00:21.628 INFO:teuthology.orchestra.run.smithi090.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60875.log.gz 2024-04-23T02:00:21.628 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.56946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45190.log 2024-04-23T02:00:21.628 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.56946.log.gz 2024-04-23T02:00:21.628 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.66260.log: gzip -5 49.6% -- replaced with /var/log/ceph/ceph-client.admin.66260.log.gz 2024-04-23T02:00:21.628 INFO:teuthology.orchestra.run.smithi090.stderr: --verbose -- /var/log/ceph/ceph-client.admin.39369.log 2024-04-23T02:00:21.628 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.45190.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45190.log.gz 2024-04-23T02:00:21.628 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30808.log 2024-04-23T02:00:21.629 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41958.log 2024-04-23T02:00:21.629 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.39369.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.30808.log: /var/log/ceph/ceph-client.admin.34008.log 2024-04-23T02:00:21.629 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39369.log.gz 2024-04-23T02:00:21.629 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30808.log.gz 2024-04-23T02:00:21.629 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49242.log 2024-04-23T02:00:21.629 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.41958.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41958.log.gz 2024-04-23T02:00:21.629 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34455.log 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34008.log.gz 2024-04-23T02:00:21.629 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-04-23T02:00:21.629 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.49242.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49242.log.gz 2024-04-23T02:00:21.629 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61546.log 2024-04-23T02:00:21.629 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.34455.log: -- /var/log/ceph/ceph-client.admin.62615.log 2024-04-23T02:00:21.630 INFO:teuthology.orchestra.run.smithi090.stderr: 50.4%/var/log/ceph/ceph-client.admin.61546.log: -- replaced with /var/log/ceph/ceph-client.admin.34455.log.gz 2024-04-23T02:00:21.630 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.61546.log.gz 2024-04-23T02:00:21.630 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30032.log 2024-04-23T02:00:21.630 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.62615.log: gzip -5 --verbose -- 49.6% -- replaced with /var/log/ceph/ceph-client.admin.62615.log.gz 2024-04-23T02:00:21.630 INFO:teuthology.orchestra.run.smithi090.stderr: /var/log/ceph/ceph-client.admin.34762.log 2024-04-23T02:00:21.630 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.30032.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30032.log.gz 2024-04-23T02:00:21.630 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33770.log 2024-04-23T02:00:21.630 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34762.log: 50.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49591.log 2024-04-23T02:00:21.630 INFO:teuthology.orchestra.run.smithi090.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34762.log.gz 2024-04-23T02:00:21.630 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50528.log 2024-04-23T02:00:21.631 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33770.log.gz 2024-04-23T02:00:21.631 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.49591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64482.log 2024-04-23T02:00:21.631 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49591.log.gz 2024-04-23T02:00:21.631 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.50528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58562.log 2024-04-23T02:00:21.631 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50528.log.gz 2024-04-23T02:00:21.631 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.64482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48210.log 2024-04-23T02:00:21.631 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.64482.log.gz 2024-04-23T02:00:21.631 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.58562.log: gzip -5 --verbose -- 49.6% -- replaced with /var/log/ceph/ceph-client.admin.58562.log.gz 2024-04-23T02:00:21.631 INFO:teuthology.orchestra.run.smithi090.stderr: /var/log/ceph/ceph-client.admin.44794.log 2024-04-23T02:00:21.631 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.48210.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48210.log.gz 2024-04-23T02:00:21.631 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31822.log 2024-04-23T02:00:21.631 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.44794.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.44794.log.gz 2024-04-23T02:00:21.631 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43869.log 2024-04-23T02:00:21.632 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33205.log 2024-04-23T02:00:21.632 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.31822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64524.log 2024-04-23T02:00:21.632 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.43869.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.31822.log.gz 2024-04-23T02:00:21.632 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43869.log.gz 2024-04-23T02:00:21.632 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67078.log 2024-04-23T02:00:21.632 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33205.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.33205.log.gz 2024-04-23T02:00:21.632 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.64524.log: gzip -5 --verbose -- 49.6% -- replaced with /var/log/ceph/ceph-client.admin.64524.log.gz 2024-04-23T02:00:21.632 INFO:teuthology.orchestra.run.smithi090.stderr: /var/log/ceph/ceph-client.admin.56737.log 2024-04-23T02:00:21.632 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.67078.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.67078.log.gz 2024-04-23T02:00:21.632 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57084.log 2024-04-23T02:00:21.632 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.56737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39178.log 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56737.log.gz 2024-04-23T02:00:21.632 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-04-23T02:00:21.633 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.57084.log: gzip 50.0% -- replaced with /var/log/ceph/ceph-client.admin.57084.log.gz 2024-04-23T02:00:21.633 INFO:teuthology.orchestra.run.smithi090.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.30270.log 2024-04-23T02:00:21.633 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.39178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38871.log 2024-04-23T02:00:21.633 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39178.log.gz 2024-04-23T02:00:21.633 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.30270.log: gzip -5 --verbose -- 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30270.log.gz 2024-04-23T02:00:21.633 INFO:teuthology.orchestra.run.smithi090.stderr: /var/log/ceph/ceph-client.admin.65306.log 2024-04-23T02:00:21.633 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38871.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38871.log.gz 2024-04-23T02:00:21.633 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32080.log 2024-04-23T02:00:21.633 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.65306.log: -- /var/log/ceph/ceph-client.admin.61813.log 2024-04-23T02:00:21.633 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.65306.log.gz 2024-04-23T02:00:21.633 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.32080.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32080.log.gz 2024-04-23T02:00:21.633 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48746.log 2024-04-23T02:00:21.634 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.61813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55247.log 2024-04-23T02:00:21.634 INFO:teuthology.orchestra.run.smithi090.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.61813.log.gz 2024-04-23T02:00:21.634 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.48746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66616.log 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48746.log.gz 2024-04-23T02:00:21.634 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-04-23T02:00:21.634 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.0.55247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42467.log 2024-04-23T02:00:21.634 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.66616.log: 50.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35549.log 2024-04-23T02:00:21.634 INFO:teuthology.orchestra.run.smithi090.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66616.log.gz 2024-04-23T02:00:21.634 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.42467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38705.log 2024-04-23T02:00:21.634 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42467.log.gz 2024-04-23T02:00:21.634 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59642.log 2024-04-23T02:00:21.634 INFO:teuthology.orchestra.run.smithi090.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.35549.log.gz 2024-04-23T02:00:21.634 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54632.log 2024-04-23T02:00:21.635 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38705.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38705.log.gz 2024-04-23T02:00:21.635 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.59642.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.59642.log.gz 2024-04-23T02:00:21.635 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64022.log 2024-04-23T02:00:21.635 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.54632.log: gzip -5 --verbose 50.0% -- /var/log/ceph/ceph-mon.a.log 2024-04-23T02:00:21.635 INFO:teuthology.orchestra.run.smithi090.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54632.log.gz 2024-04-23T02:00:21.635 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.64022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34624.log 2024-04-23T02:00:21.635 INFO:teuthology.orchestra.run.smithi090.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.64022.log.gz 2024-04-23T02:00:21.635 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51161.log 2024-04-23T02:00:21.635 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.34624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57898.log 2024-04-23T02:00:21.635 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34624.log.gz 2024-04-23T02:00:21.635 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.51161.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51161.log.gz 2024-04-23T02:00:21.635 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63406.log 2024-04-23T02:00:21.635 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34099.log 2024-04-23T02:00:21.636 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.63406.log: /var/log/ceph/ceph-client.admin.57898.log: 50.0% 50.0% -- replaced with /var/log/ceph/ceph-client.admin.63406.log.gz 2024-04-23T02:00:21.636 INFO:teuthology.orchestra.run.smithi090.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57898.log.gz 2024-04-23T02:00:21.636 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66444.log 2024-04-23T02:00:21.636 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35676.log 2024-04-23T02:00:21.636 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34099.log.gz 2024-04-23T02:00:21.636 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.66444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52405.log 2024-04-23T02:00:21.636 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.66444.log.gz 2024-04-23T02:00:21.636 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32129.log 2024-04-23T02:00:21.636 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0%/var/log/ceph/ceph-client.admin.52405.log: -- replaced with /var/log/ceph/ceph-client.admin.35676.log.gz 2024-04-23T02:00:21.636 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.52405.log.gz 2024-04-23T02:00:21.637 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60270.log 2024-04-23T02:00:21.637 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.32129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65566.log 2024-04-23T02:00:21.637 INFO:teuthology.orchestra.run.smithi090.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.32129.log.gz 2024-04-23T02:00:21.637 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.60270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36383.log 2024-04-23T02:00:21.637 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.60270.log.gz 2024-04-23T02:00:21.637 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.65566.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.65566.log.gz 2024-04-23T02:00:21.637 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66658.log 2024-04-23T02:00:21.637 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.36383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41789.log 2024-04-23T02:00:21.637 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36383.log.gz 2024-04-23T02:00:21.637 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.66658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48528.log 2024-04-23T02:00:21.637 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.41789.log: 49.6% 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41789.log.gz 2024-04-23T02:00:21.637 INFO:teuthology.orchestra.run.smithi090.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66658.log.gz 2024-04-23T02:00:21.637 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34802.log 2024-04-23T02:00:21.638 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.48528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50395.log 2024-04-23T02:00:21.638 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48528.log.gz 2024-04-23T02:00:21.638 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51494.log 2024-04-23T02:00:21.638 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.50395.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34802.log.gz 2024-04-23T02:00:21.638 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50395.log.gz 2024-04-23T02:00:21.638 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54125.log 2024-04-23T02:00:21.638 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.51494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49095.log 2024-04-23T02:00:21.638 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51494.log.gz 2024-04-23T02:00:21.638 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.54125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61256.log 2024-04-23T02:00:21.638 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.49095.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.54125.log.gz 2024-04-23T02:00:21.639 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49095.log.gz 2024-04-23T02:00:21.639 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55444.log 2024-04-23T02:00:21.639 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.61256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62073.log 2024-04-23T02:00:21.639 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.61256.log.gz 2024-04-23T02:00:21.639 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.55444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56548.log 2024-04-23T02:00:21.639 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.62073.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.55444.log.gz 2024-04-23T02:00:21.639 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.62073.log.gz 2024-04-23T02:00:21.639 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62890.log 2024-04-23T02:00:21.639 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.56548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46231.log 2024-04-23T02:00:21.639 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56548.log.gz 2024-04-23T02:00:21.639 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.62890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44116.log 2024-04-23T02:00:21.639 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.46231.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.62890.log.gz 2024-04-23T02:00:21.639 INFO:teuthology.orchestra.run.smithi090.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.46231.log.gz 2024-04-23T02:00:21.640 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34363.log 2024-04-23T02:00:21.640 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.44116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44276.log 2024-04-23T02:00:21.640 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44116.log.gz 2024-04-23T02:00:21.640 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60427.log 2024-04-23T02:00:21.640 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.44276.log: 49.6% 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44276.log.gz 2024-04-23T02:00:21.640 INFO:teuthology.orchestra.run.smithi090.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34363.log.gz 2024-04-23T02:00:21.640 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52506.log 2024-04-23T02:00:21.640 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.0.60427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61700.log 2024-04-23T02:00:21.640 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.52506.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.52506.log.gz 2024-04-23T02:00:21.640 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38333.log 2024-04-23T02:00:21.640 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.61700.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.61700.log.gz 2024-04-23T02:00:21.640 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61904.log 2024-04-23T02:00:21.641 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38333.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38333.log.gz 2024-04-23T02:00:21.641 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38751.log 2024-04-23T02:00:21.641 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.61904.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.61904.log.gz 2024-04-23T02:00:21.641 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28423.log 2024-04-23T02:00:21.641 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38751.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38751.log.gz 2024-04-23T02:00:21.641 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42831.log 2024-04-23T02:00:21.641 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.28423.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28423.log.gz 2024-04-23T02:00:21.641 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55850.log 2024-04-23T02:00:21.641 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.42831.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.42831.log.gz 2024-04-23T02:00:21.641 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28053.log 2024-04-23T02:00:21.641 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.55850.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55850.log.gz 2024-04-23T02:00:21.641 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59077.log 2024-04-23T02:00:21.642 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.28053.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.28053.log.gz 2024-04-23T02:00:21.642 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43620.log 2024-04-23T02:00:21.642 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.59077.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.59077.log.gz 2024-04-23T02:00:21.642 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42789.log 2024-04-23T02:00:21.642 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.43620.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43620.log.gz 2024-04-23T02:00:21.642 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28424.log 2024-04-23T02:00:21.642 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.42789.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42789.log.gz 2024-04-23T02:00:21.642 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36523.log 2024-04-23T02:00:21.642 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.28424.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28424.log.gz 2024-04-23T02:00:21.642 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43173.log 2024-04-23T02:00:21.642 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.36523.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36523.log.gz 2024-04-23T02:00:21.642 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47863.log 2024-04-23T02:00:21.643 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.43173.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.43173.log.gz 2024-04-23T02:00:21.643 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60791.log 2024-04-23T02:00:21.643 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.47863.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47863.log.gz 2024-04-23T02:00:21.643 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54761.log 2024-04-23T02:00:21.643 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.60791.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.60791.log.gz 2024-04-23T02:00:21.643 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63217.log 2024-04-23T02:00:21.643 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.54761.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.54761.log.gz 2024-04-23T02:00:21.643 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60462.log 2024-04-23T02:00:21.643 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.63217.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.63217.log.gz 2024-04-23T02:00:21.643 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51445.log 2024-04-23T02:00:21.643 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.60462.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.60462.log.gz 2024-04-23T02:00:21.643 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43264.log 2024-04-23T02:00:21.643 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.51445.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51445.log.gz 2024-04-23T02:00:21.644 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61505.log 2024-04-23T02:00:21.644 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.43264.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43264.log.gz 2024-04-23T02:00:21.644 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58645.log 2024-04-23T02:00:21.644 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.61505.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.61505.log.gz 2024-04-23T02:00:21.644 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47545.log 2024-04-23T02:00:21.644 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.58645.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.58645.log.gz 2024-04-23T02:00:21.644 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58859.log 2024-04-23T02:00:21.644 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.47545.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47545.log.gz 2024-04-23T02:00:21.644 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66868.log 2024-04-23T02:00:21.644 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.58859.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.58859.log.gz 2024-04-23T02:00:21.644 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31922.log 2024-04-23T02:00:21.645 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.66868.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.66868.log.gz 2024-04-23T02:00:21.645 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55582.log 2024-04-23T02:00:21.645 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.31922.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31922.log.gz 2024-04-23T02:00:21.645 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29305.log 2024-04-23T02:00:21.645 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.55582.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.55582.log.gz 2024-04-23T02:00:21.645 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42425.log 2024-04-23T02:00:21.645 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.29305.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.29305.log.gz 2024-04-23T02:00:21.645 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38791.log 2024-04-23T02:00:21.645 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.42425.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42425.log.gz 2024-04-23T02:00:21.645 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57805.log 2024-04-23T02:00:21.645 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38791.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.38791.log.gz 2024-04-23T02:00:21.645 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33917.log 2024-04-23T02:00:21.646 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.57805.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.57805.log.gz 2024-04-23T02:00:21.646 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50703.log 2024-04-23T02:00:21.646 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33917.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33917.log.gz 2024-04-23T02:00:21.646 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43849.log 2024-04-23T02:00:21.646 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.50703.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50703.log.gz 2024-04-23T02:00:21.646 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37886.log 2024-04-23T02:00:21.646 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.43849.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43849.log.gz 2024-04-23T02:00:21.646 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55196.log 2024-04-23T02:00:21.646 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37886.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37886.log.gz 2024-04-23T02:00:21.646 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36243.log 2024-04-23T02:00:21.646 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.55196.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55196.log.gz 2024-04-23T02:00:21.647 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62790.log 2024-04-23T02:00:21.647 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.36243.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36243.log.gz 2024-04-23T02:00:21.647 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60651.log 2024-04-23T02:00:21.647 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.62790.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.62790.log.gz 2024-04-23T02:00:21.647 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52110.log 2024-04-23T02:00:21.647 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.60651.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.60651.log.gz 2024-04-23T02:00:21.647 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47594.log 2024-04-23T02:00:21.647 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.52110.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.52110.log.gz 2024-04-23T02:00:21.647 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51743.log 2024-04-23T02:00:21.647 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.47594.log: 47.6% -- replaced with /var/log/ceph/ceph-client.admin.47594.log.gz 2024-04-23T02:00:21.647 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45101.log 2024-04-23T02:00:21.647 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.51743.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51743.log.gz 2024-04-23T02:00:21.647 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58380.log 2024-04-23T02:00:21.648 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.45101.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45101.log.gz 2024-04-23T02:00:21.648 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34713.log 2024-04-23T02:00:21.648 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.58380.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.58380.log.gz 2024-04-23T02:00:21.648 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37634.log 2024-04-23T02:00:21.648 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34713.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34713.log.gz 2024-04-23T02:00:21.648 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52635.log 2024-04-23T02:00:21.648 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37634.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37634.log.gz 2024-04-23T02:00:21.648 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52764.log 2024-04-23T02:00:21.648 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.52635.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.52635.log.gz 2024-04-23T02:00:21.648 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48250.log 2024-04-23T02:00:21.648 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.52764.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.52764.log.gz 2024-04-23T02:00:21.649 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38382.log 2024-04-23T02:00:21.649 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.48250.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48250.log.gz 2024-04-23T02:00:21.649 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59722.log 2024-04-23T02:00:21.649 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38382.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38382.log.gz 2024-04-23T02:00:21.649 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37110.log 2024-04-23T02:00:21.649 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.59722.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.59722.log.gz 2024-04-23T02:00:21.649 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55155.log 2024-04-23T02:00:21.649 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37110.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37110.log.gz 2024-04-23T02:00:21.649 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66826.log 2024-04-23T02:00:21.649 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.55155.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55155.log.gz 2024-04-23T02:00:21.649 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59553.log 2024-04-23T02:00:21.649 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.66826.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.66826.log.gz 2024-04-23T02:00:21.650 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30361.log 2024-04-23T02:00:21.650 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.59553.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.59553.log.gz 2024-04-23T02:00:21.650 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29500.log 2024-04-23T02:00:21.650 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.30361.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30361.log.gz 2024-04-23T02:00:21.650 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28996.log 2024-04-23T02:00:21.650 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.29500.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29500.log.gz 2024-04-23T02:00:21.650 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42964.log 2024-04-23T02:00:21.650 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.28996.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28996.log.gz 2024-04-23T02:00:21.650 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39247.log 2024-04-23T02:00:21.650 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.42964.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42964.log.gz 2024-04-23T02:00:21.650 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62830.log 2024-04-23T02:00:21.650 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.39247.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39247.log.gz 2024-04-23T02:00:21.651 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46767.log 2024-04-23T02:00:21.651 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.62830.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.62830.log.gz 2024-04-23T02:00:21.651 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40208.log 2024-04-23T02:00:21.651 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.46767.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46767.log.gz 2024-04-23T02:00:21.651 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37243.log 2024-04-23T02:00:21.651 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.40208.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40208.log.gz 2024-04-23T02:00:21.651 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43451.log 2024-04-23T02:00:21.651 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37243.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.37243.log.gz 2024-04-23T02:00:21.651 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51310.log 2024-04-23T02:00:21.651 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.43451.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43451.log.gz 2024-04-23T02:00:21.651 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34277.log 2024-04-23T02:00:21.651 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.51310.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51310.log.gz 2024-04-23T02:00:21.652 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43362.log 2024-04-23T02:00:21.652 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34277.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34277.log.gz 2024-04-23T02:00:21.652 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32311.log 2024-04-23T02:00:21.652 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.43362.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43362.log.gz 2024-04-23T02:00:21.652 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33247.log 2024-04-23T02:00:21.652 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.32311.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.32311.log.gz 2024-04-23T02:00:21.652 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58695.log 2024-04-23T02:00:21.652 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33247.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33247.log.gz 2024-04-23T02:00:21.652 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62575.log 2024-04-23T02:00:21.652 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.58695.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.58695.log.gz 2024-04-23T02:00:21.652 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39009.log 2024-04-23T02:00:21.652 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.62575.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.62575.log.gz 2024-04-23T02:00:21.653 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65347.log 2024-04-23T02:00:21.653 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.39009.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39009.log.gz 2024-04-23T02:00:21.653 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59952.log 2024-04-23T02:00:21.653 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.65347.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.65347.log.gz 2024-04-23T02:00:21.653 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66084.log 2024-04-23T02:00:21.653 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.59952.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.59952.log.gz 2024-04-23T02:00:21.653 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36054.log 2024-04-23T02:00:21.653 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.66084.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.66084.log.gz 2024-04-23T02:00:21.653 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64739.log 2024-04-23T02:00:21.653 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.36054.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36054.log.gz 2024-04-23T02:00:21.653 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48568.log 2024-04-23T02:00:21.653 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.64739.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.64739.log.gz 2024-04-23T02:00:21.653 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40986.log 2024-04-23T02:00:21.654 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.48568.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48568.log.gz 2024-04-23T02:00:21.654 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37738.log 2024-04-23T02:00:21.654 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.40986.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40986.log.gz 2024-04-23T02:00:21.654 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60700.log 2024-04-23T02:00:21.654 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37738.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37738.log.gz 2024-04-23T02:00:21.654 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63455.log 2024-04-23T02:00:21.654 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.60700.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.60700.log.gz 2024-04-23T02:00:21.654 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55493.log 2024-04-23T02:00:21.654 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.63455.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.63455.log.gz 2024-04-23T02:00:21.654 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62302.log 2024-04-23T02:00:21.654 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.55493.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55493.log.gz 2024-04-23T02:00:21.654 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27732.log 2024-04-23T02:00:21.654 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.62302.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.62302.log.gz 2024-04-23T02:00:21.655 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50786.log 2024-04-23T02:00:21.655 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.27732.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.27732.log.gz 2024-04-23T02:00:21.655 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56641.log 2024-04-23T02:00:21.655 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.50786.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50786.log.gz 2024-04-23T02:00:21.655 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66259.log 2024-04-23T02:00:21.655 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.56641.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.56641.log.gz 2024-04-23T02:00:21.655 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57671.log 2024-04-23T02:00:21.655 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.66259.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.66259.log.gz 2024-04-23T02:00:21.655 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29951.log 2024-04-23T02:00:21.655 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.57671.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.57671.log.gz 2024-04-23T02:00:21.655 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56688.log 2024-04-23T02:00:21.655 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.29951.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29951.log.gz 2024-04-23T02:00:21.656 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64411.log 2024-04-23T02:00:21.656 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.56688.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.56688.log.gz 2024-04-23T02:00:21.656 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32171.log 2024-04-23T02:00:21.656 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.64411.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.64411.log.gz 2024-04-23T02:00:21.656 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56201.log 2024-04-23T02:00:21.656 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.32171.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32171.log.gz 2024-04-23T02:00:21.656 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27591.log 2024-04-23T02:00:21.656 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.56201.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56201.log.gz 2024-04-23T02:00:21.656 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43004.log 2024-04-23T02:00:21.656 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.27591.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.27591.log.gz 2024-04-23T02:00:21.656 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35069.log 2024-04-23T02:00:21.656 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.43004.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43004.log.gz 2024-04-23T02:00:21.657 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34406.log 2024-04-23T02:00:21.657 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35069.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.35069.log.gz 2024-04-23T02:00:21.657 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47236.log 2024-04-23T02:00:21.657 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34406.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34406.log.gz 2024-04-23T02:00:21.657 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56290.log 2024-04-23T02:00:21.657 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.47236.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47236.log.gz 2024-04-23T02:00:21.657 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48390.log 2024-04-23T02:00:21.657 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.56290.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56290.log.gz 2024-04-23T02:00:21.657 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46280.log 2024-04-23T02:00:21.657 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.48390.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48390.log.gz 2024-04-23T02:00:21.657 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64565.log 2024-04-23T02:00:21.657 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.46280.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46280.log.gz 2024-04-23T02:00:21.658 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40895.log 2024-04-23T02:00:21.658 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.64565.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.64565.log.gz 2024-04-23T02:00:21.658 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54672.log 2024-04-23T02:00:21.658 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.40895.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40895.log.gz 2024-04-23T02:00:21.658 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54374.log 2024-04-23T02:00:21.658 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.54672.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.54672.log.gz 2024-04-23T02:00:21.658 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54196.log 2024-04-23T02:00:21.658 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.54374.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54374.log.gz 2024-04-23T02:00:21.658 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32577.log 2024-04-23T02:00:21.658 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.54196.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.54196.log.gz 2024-04-23T02:00:21.658 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59206.log 2024-04-23T02:00:21.658 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.32577.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.32577.log.gz 2024-04-23T02:00:21.659 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41126.log 2024-04-23T02:00:21.659 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.59206.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.59206.log.gz 2024-04-23T02:00:21.659 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44018.log 2024-04-23T02:00:21.659 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.41126.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41126.log.gz 2024-04-23T02:00:21.659 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66700.log 2024-04-23T02:00:21.659 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.44018.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44018.log.gz 2024-04-23T02:00:21.659 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41838.log 2024-04-23T02:00:21.659 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.66700.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.66700.log.gz 2024-04-23T02:00:21.659 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.33413.log 2024-04-23T02:00:21.659 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.41838.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41838.log.gz 2024-04-23T02:00:21.659 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34971.log 2024-04-23T02:00:21.659 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.0.33413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59246.log 2024-04-23T02:00:21.660 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34971.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34971.log.gz 2024-04-23T02:00:21.660 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63008.log 2024-04-23T02:00:21.660 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.59246.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.59246.log.gz 2024-04-23T02:00:21.660 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61463.log 2024-04-23T02:00:21.660 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.63008.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.63008.log.gz 2024-04-23T02:00:21.660 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41518.log 2024-04-23T02:00:21.660 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.61463.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.61463.log.gz 2024-04-23T02:00:21.660 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51228.log 2024-04-23T02:00:21.660 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.41518.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41518.log.gz 2024-04-23T02:00:21.660 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48589.log 2024-04-23T02:00:21.660 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.51228.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.51228.log.gz 2024-04-23T02:00:21.660 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51095.log 2024-04-23T02:00:21.660 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.48589.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48589.log.gz 2024-04-23T02:00:21.661 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33819.log 2024-04-23T02:00:21.661 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.51095.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51095.log.gz 2024-04-23T02:00:21.661 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30081.log 2024-04-23T02:00:21.661 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33819.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33819.log.gz 2024-04-23T02:00:21.665 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55110.log 2024-04-23T02:00:21.665 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.30081.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.30081.log.gz 2024-04-23T02:00:21.665 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35865.log 2024-04-23T02:00:21.665 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.55110.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55110.log.gz 2024-04-23T02:00:21.665 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65126.log 2024-04-23T02:00:21.666 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35865.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.35865.log.gz 2024-04-23T02:00:21.666 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52288.log 2024-04-23T02:00:21.666 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.65126.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.65126.log.gz 2024-04-23T02:00:21.666 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45408.log 2024-04-23T02:00:21.666 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.52288.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.52288.log.gz 2024-04-23T02:00:21.666 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48975.log 2024-04-23T02:00:21.666 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.45408.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45408.log.gz 2024-04-23T02:00:21.666 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48348.log 2024-04-23T02:00:21.666 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.48975.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48975.log.gz 2024-04-23T02:00:21.666 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56419.log 2024-04-23T02:00:21.667 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.48348.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48348.log.gz 2024-04-23T02:00:21.667 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54294.log 2024-04-23T02:00:21.667 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.56419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56161.log 2024-04-23T02:00:21.673 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.54294.log: 49.6% 50.4% -- replaced with /var/log/ceph/ceph-client.admin.56419.log.gz 2024-04-23T02:00:21.679 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58737.log 2024-04-23T02:00:21.680 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.56161.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56161.log.gz 2024-04-23T02:00:21.680 INFO:teuthology.orchestra.run.smithi090.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54294.log.gz 2024-04-23T02:00:21.686 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44654.log 2024-04-23T02:00:21.700 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.58737.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.58737.log.gz 2024-04-23T02:00:21.700 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29036.log 2024-04-23T02:00:21.700 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.44654.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.44654.log.gz 2024-04-23T02:00:21.758 INFO:teuthology.orchestra.run.smithi090.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.0.46174.log.gz 2024-04-23T02:00:21.758 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40846.log 2024-04-23T02:00:21.758 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.29036.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.29036.log.gz 2024-04-23T02:00:21.758 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34842.log 2024-04-23T02:00:21.758 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.40846.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40846.log.gz 2024-04-23T02:00:21.758 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27507.log 2024-04-23T02:00:21.758 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34842.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34842.log.gz 2024-04-23T02:00:21.759 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47823.log 2024-04-23T02:00:21.759 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.27507.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.27507.log.gz 2024-04-23T02:00:21.759 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52328.log 2024-04-23T02:00:21.759 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.47823.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47823.log.gz 2024-04-23T02:00:21.759 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31989.log 2024-04-23T02:00:21.759 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.52328.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.52328.log.gz 2024-04-23T02:00:21.759 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47147.log 2024-04-23T02:00:21.759 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.31989.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31989.log.gz 2024-04-23T02:00:21.759 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41698.log 2024-04-23T02:00:21.759 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.47147.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47147.log.gz 2024-04-23T02:00:21.759 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43044.log 2024-04-23T02:00:21.759 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.41698.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41698.log.gz 2024-04-23T02:00:21.760 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41224.log 2024-04-23T02:00:21.760 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.43044.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43044.log.gz 2024-04-23T02:00:21.760 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49800.log 2024-04-23T02:00:21.760 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.41224.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41224.log.gz 2024-04-23T02:00:21.760 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52595.log 2024-04-23T02:00:21.760 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.49800.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49800.log.gz 2024-04-23T02:00:21.760 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42656.log 2024-04-23T02:00:21.760 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.52595.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.52595.log.gz 2024-04-23T02:00:21.760 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43491.log 2024-04-23T02:00:21.760 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.42656.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42656.log.gz 2024-04-23T02:00:21.761 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33470.log 2024-04-23T02:00:21.761 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.43491.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43491.log.gz 2024-04-23T02:00:21.761 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64349.log 2024-04-23T02:00:21.761 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33470.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33470.log.gz 2024-04-23T02:00:21.761 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41084.log 2024-04-23T02:00:21.761 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.64349.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.64349.log.gz 2024-04-23T02:00:21.761 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43124.log 2024-04-23T02:00:21.761 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.41084.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41084.log.gz 2024-04-23T02:00:21.761 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51403.log 2024-04-23T02:00:21.761 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.43124.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43124.log.gz 2024-04-23T02:00:21.761 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44928.log 2024-04-23T02:00:21.761 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.51403.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51403.log.gz 2024-04-23T02:00:21.761 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63853.log 2024-04-23T02:00:21.762 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.44928.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44928.log.gz 2024-04-23T02:00:21.762 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56339.log 2024-04-23T02:00:21.762 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.63853.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.63853.log.gz 2024-04-23T02:00:21.762 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47992.log 2024-04-23T02:00:21.762 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.56339.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.56339.log.gz 2024-04-23T02:00:21.762 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37696.log 2024-04-23T02:00:21.762 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.47992.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.47992.log.gz 2024-04-23T02:00:21.762 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65818.log 2024-04-23T02:00:21.762 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37696.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37696.log.gz 2024-04-23T02:00:21.762 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49055.log 2024-04-23T02:00:21.762 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.65818.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.65818.log.gz 2024-04-23T02:00:21.763 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36712.log 2024-04-23T02:00:21.763 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.49055.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49055.log.gz 2024-04-23T02:00:21.763 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36194.log 2024-04-23T02:00:21.763 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.36712.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36712.log.gz 2024-04-23T02:00:21.763 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63624.log 2024-04-23T02:00:21.763 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.36194.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36194.log.gz 2024-04-23T02:00:21.763 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28137.log 2024-04-23T02:00:21.763 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.63624.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.63624.log.gz 2024-04-23T02:00:21.763 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49015.log 2024-04-23T02:00:21.763 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.28137.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28137.log.gz 2024-04-23T02:00:21.763 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47903.log 2024-04-23T02:00:21.763 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.49015.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49015.log.gz 2024-04-23T02:00:21.764 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36425.log 2024-04-23T02:00:21.764 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.47903.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47903.log.gz 2024-04-23T02:00:21.764 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32031.log 2024-04-23T02:00:21.764 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.36425.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36425.log.gz 2024-04-23T02:00:21.764 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59384.log 2024-04-23T02:00:21.764 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.32031.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.32031.log.gz 2024-04-23T02:00:21.764 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40806.log 2024-04-23T02:00:21.764 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.59384.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.59384.log.gz 2024-04-23T02:00:21.764 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44196.log 2024-04-23T02:00:21.764 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.40806.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.40806.log.gz 2024-04-23T02:00:21.764 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66742.log 2024-04-23T02:00:21.764 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.44196.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44196.log.gz 2024-04-23T02:00:21.764 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64300.log 2024-04-23T02:00:21.765 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.66742.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.66742.log.gz 2024-04-23T02:00:21.765 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46678.log 2024-04-23T02:00:21.765 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.64300.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.64300.log.gz 2024-04-23T02:00:21.765 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40027.log 2024-04-23T02:00:21.765 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.46678.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46678.log.gz 2024-04-23T02:00:21.765 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36005.log 2024-04-23T02:00:21.765 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.40027.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40027.log.gz 2024-04-23T02:00:21.765 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41641.log 2024-04-23T02:00:21.765 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.36005.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36005.log.gz 2024-04-23T02:00:21.766 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29733.log 2024-04-23T02:00:21.767 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.0.41641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31684.log 2024-04-23T02:00:21.767 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.29733.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29733.log.gz 2024-04-23T02:00:21.780 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32220.log 2024-04-23T02:00:21.780 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.31684.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31684.log.gz 2024-04-23T02:00:21.786 INFO:teuthology.orchestra.run.smithi090.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.0.28592.log.gz 2024-04-23T02:00:21.786 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45577.log 2024-04-23T02:00:21.786 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.32220.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.32220.log.gz 2024-04-23T02:00:21.786 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61596.log 2024-04-23T02:00:21.786 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.45577.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45577.log.gz 2024-04-23T02:00:21.786 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33065.log 2024-04-23T02:00:21.803 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.61596.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.61596.log.gz 2024-04-23T02:00:21.803 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36474.log 2024-04-23T02:00:21.803 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33065.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33065.log.gz 2024-04-23T02:00:21.803 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42285.log 2024-04-23T02:00:21.803 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.36474.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36474.log.gz 2024-04-23T02:00:21.804 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61015.log 2024-04-23T02:00:21.804 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.42285.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42285.log.gz 2024-04-23T02:00:21.804 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39049.log 2024-04-23T02:00:21.804 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.61015.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.61015.log.gz 2024-04-23T02:00:21.804 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62484.log 2024-04-23T02:00:21.804 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.39049.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.39049.log.gz 2024-04-23T02:00:21.804 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31128.log 2024-04-23T02:00:21.804 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.62484.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.62484.log.gz 2024-04-23T02:00:21.804 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46371.log 2024-04-23T02:00:21.804 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.31128.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31128.log.gz 2024-04-23T02:00:21.804 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30997.log 2024-04-23T02:00:21.804 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.46371.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46371.log.gz 2024-04-23T02:00:21.804 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47923.log 2024-04-23T02:00:21.805 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.30997.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30997.log.gz 2024-04-23T02:00:21.805 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27902.log 2024-04-23T02:00:21.805 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.47923.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47923.log.gz 2024-04-23T02:00:21.805 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41569.log 2024-04-23T02:00:21.805 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.27902.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.27902.log.gz 2024-04-23T02:00:21.805 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57184.log 2024-04-23T02:00:21.805 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.41569.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41569.log.gz 2024-04-23T02:00:21.805 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28422.log 2024-04-23T02:00:21.805 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.57184.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.57184.log.gz 2024-04-23T02:00:21.806 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59464.log 2024-04-23T02:00:21.806 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.28422.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.28422.log.gz 2024-04-23T02:00:21.806 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45942.log 2024-04-23T02:00:21.806 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.59464.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.59464.log.gz 2024-04-23T02:00:21.806 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64160.log 2024-04-23T02:00:21.806 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.45942.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45942.log.gz 2024-04-23T02:00:21.825 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63664.log 2024-04-23T02:00:21.825 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.64160.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.64160.log.gz 2024-04-23T02:00:21.825 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60153.log 2024-04-23T02:00:21.825 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.63664.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.63664.log.gz 2024-04-23T02:00:21.825 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67207.log 2024-04-23T02:00:21.825 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.60153.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.60153.log.gz 2024-04-23T02:00:21.825 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35467.log 2024-04-23T02:00:21.825 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.67207.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.67207.log.gz 2024-04-23T02:00:21.825 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60924.log 2024-04-23T02:00:21.825 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35467.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35467.log.gz 2024-04-23T02:00:21.826 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45983.log 2024-04-23T02:00:21.826 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.60924.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.60924.log.gz 2024-04-23T02:00:21.826 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35487.log 2024-04-23T02:00:21.826 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.45983.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45983.log.gz 2024-04-23T02:00:21.826 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39929.log 2024-04-23T02:00:21.826 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35487.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35487.log.gz 2024-04-23T02:00:21.826 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51543.log 2024-04-23T02:00:21.826 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.39929.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39929.log.gz 2024-04-23T02:00:21.826 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31317.log 2024-04-23T02:00:21.826 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.51543.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51543.log.gz 2024-04-23T02:00:21.826 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38969.log 2024-04-23T02:00:21.826 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.31317.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.31317.log.gz 2024-04-23T02:00:21.826 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29822.log 2024-04-23T02:00:21.826 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38969.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38969.log.gz 2024-04-23T02:00:21.827 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63097.log 2024-04-23T02:00:21.827 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.29822.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29822.log.gz 2024-04-23T02:00:21.827 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63942.log 2024-04-23T02:00:21.827 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.63097.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.63097.log.gz 2024-04-23T02:00:21.827 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38831.log 2024-04-23T02:00:21.827 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.63942.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.63942.log.gz 2024-04-23T02:00:21.827 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50458.log 2024-04-23T02:00:21.827 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38831.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38831.log.gz 2024-04-23T02:00:21.827 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37828.log 2024-04-23T02:00:21.827 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.50458.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50458.log.gz 2024-04-23T02:00:21.827 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31297.log 2024-04-23T02:00:21.835 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.0.37828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33650.log 2024-04-23T02:00:21.835 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.31297.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31297.log.gz 2024-04-23T02:00:21.839 INFO:teuthology.orchestra.run.smithi090.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.0.55247.log.gz 2024-04-23T02:00:21.919 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54884.log 2024-04-23T02:00:21.920 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33650.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.33650.log.gz 2024-04-23T02:00:21.920 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35914.log 2024-04-23T02:00:21.920 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.54884.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54884.log.gz 2024-04-23T02:00:21.920 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36334.log 2024-04-23T02:00:21.920 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35914.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35914.log.gz 2024-04-23T02:00:21.920 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53609.log 2024-04-23T02:00:21.920 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.36334.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.36334.log.gz 2024-04-23T02:00:21.920 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33134.log 2024-04-23T02:00:21.920 INFO:teuthology.orchestra.run.smithi090.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.0.60427.log.gz 2024-04-23T02:00:21.920 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.53609.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53609.log.gz 2024-04-23T02:00:21.920 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66952.log 2024-04-23T02:00:21.920 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.33134.log: -- /var/log/ceph/ceph-client.admin.43531.log 2024-04-23T02:00:21.921 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33134.log.gz 2024-04-23T02:00:21.921 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.66952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34584.log 2024-04-23T02:00:21.921 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.66952.log.gz 2024-04-23T02:00:21.921 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.43531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30410.log 2024-04-23T02:00:21.921 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43531.log.gz/var/log/ceph/ceph-client.admin.34584.log: 2024-04-23T02:00:21.921 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34584.log.gz 2024-04-23T02:00:21.921 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36095.log 2024-04-23T02:00:21.921 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45715.log 2024-04-23T02:00:21.921 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.30410.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.30410.log.gz 2024-04-23T02:00:21.921 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.36095.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36095.log.gz 2024-04-23T02:00:21.921 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35189.log 2024-04-23T02:00:21.921 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28956.log 2024-04-23T02:00:21.921 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35189.log: /var/log/ceph/ceph-client.admin.45715.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45715.log.gz 2024-04-23T02:00:21.921 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35189.log.gz 2024-04-23T02:00:21.922 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60973.log 2024-04-23T02:00:21.922 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59602.log 2024-04-23T02:00:21.922 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.28956.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28956.log.gz 2024-04-23T02:00:21.922 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.60973.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.60973.log.gz 2024-04-23T02:00:21.922 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43313.log 2024-04-23T02:00:21.922 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35327.log 2024-04-23T02:00:21.922 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.43313.log: /var/log/ceph/ceph-client.admin.59602.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.59602.log.gz 2024-04-23T02:00:21.922 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43313.log.gz 2024-04-23T02:00:21.922 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55060.log 2024-04-23T02:00:21.922 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37548.log 2024-04-23T02:00:21.922 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35327.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35327.log.gz 2024-04-23T02:00:21.922 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.55060.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55060.log.gz 2024-04-23T02:00:21.922 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50436.log 2024-04-23T02:00:21.923 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37798.log 2024-04-23T02:00:21.923 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37548.log: /var/log/ceph/ceph-client.admin.50436.log: 50.0% 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37548.log.gz 2024-04-23T02:00:21.923 INFO:teuthology.orchestra.run.smithi090.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50436.log.gz 2024-04-23T02:00:21.923 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37263.log 2024-04-23T02:00:21.923 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53160.log 2024-04-23T02:00:21.923 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37798.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.37798.log.gz 2024-04-23T02:00:21.923 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37263.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37263.log.gz 2024-04-23T02:00:21.923 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44465.log 2024-04-23T02:00:21.923 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55364.log 2024-04-23T02:00:21.923 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.44465.log: /var/log/ceph/ceph-client.admin.53160.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.53160.log.gz 2024-04-23T02:00:21.923 INFO:teuthology.orchestra.run.smithi090.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.44465.log.gz 2024-04-23T02:00:21.923 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35109.log 2024-04-23T02:00:21.923 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39887.log 2024-04-23T02:00:21.924 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.55364.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55364.log.gz 2024-04-23T02:00:21.924 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35109.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.35109.log.gz 2024-04-23T02:00:21.924 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38156.log 2024-04-23T02:00:21.924 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48130.log 2024-04-23T02:00:21.924 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38156.log: /var/log/ceph/ceph-client.admin.39887.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39887.log.gz 2024-04-23T02:00:21.924 INFO:teuthology.orchestra.run.smithi090.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.38156.log.gz 2024-04-23T02:00:21.924 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50826.log 2024-04-23T02:00:21.924 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65608.log 2024-04-23T02:00:21.924 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.48130.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48130.log.gz 2024-04-23T02:00:21.924 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.50826.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50826.log.gz 2024-04-23T02:00:21.924 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39319.log 2024-04-23T02:00:21.924 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53965.log 2024-04-23T02:00:21.924 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.39319.log: /var/log/ceph/ceph-client.admin.65608.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.65608.log.gz 2024-04-23T02:00:21.925 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39319.log.gz 2024-04-23T02:00:21.925 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46818.log 2024-04-23T02:00:21.925 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33114.log 2024-04-23T02:00:21.925 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.53965.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53965.log.gz 2024-04-23T02:00:21.925 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.46818.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46818.log.gz 2024-04-23T02:00:21.925 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29871.log 2024-04-23T02:00:21.925 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58491.log 2024-04-23T02:00:21.925 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.29871.log: /var/log/ceph/ceph-client.admin.33114.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33114.log.gz 2024-04-23T02:00:21.925 INFO:teuthology.orchestra.run.smithi090.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.29871.log.gz 2024-04-23T02:00:21.925 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55808.log 2024-04-23T02:00:21.925 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39509.log 2024-04-23T02:00:21.925 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.58491.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.58491.log.gz 2024-04-23T02:00:21.925 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.55808.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55808.log.gz 2024-04-23T02:00:21.926 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62162.log 2024-04-23T02:00:21.926 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54979.log 2024-04-23T02:00:21.926 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.62162.log: /var/log/ceph/ceph-client.admin.39509.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.39509.log.gz 2024-04-23T02:00:21.926 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.62162.log.gz 2024-04-23T02:00:21.926 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31515.log 2024-04-23T02:00:21.926 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52715.log 2024-04-23T02:00:21.926 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.54979.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54979.log.gz 2024-04-23T02:00:21.926 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.31515.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31515.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65919.log 2024-04-23T02:00:21.926 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-04-23T02:00:21.926 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66574.log 2024-04-23T02:00:21.926 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.52715.log: /var/log/ceph/ceph-client.admin.65919.log: 50.0% 49.6% -- replaced with /var/log/ceph/ceph-client.admin.52715.log.gz 2024-04-23T02:00:21.926 INFO:teuthology.orchestra.run.smithi090.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65919.log.gz 2024-04-23T02:00:21.926 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32449.log 2024-04-23T02:00:21.926 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35816.log 2024-04-23T02:00:21.927 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.66574.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.66574.log.gz 2024-04-23T02:00:21.927 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.32449.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.32449.log.gz 2024-04-23T02:00:21.927 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30459.log 2024-04-23T02:00:21.927 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28011.log 2024-04-23T02:00:21.927 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35816.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35816.log.gz 2024-04-23T02:00:21.927 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.30459.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.30459.log.gz 2024-04-23T02:00:21.927 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56241.log 2024-04-23T02:00:21.927 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37438.log 2024-04-23T02:00:21.927 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.56241.log: /var/log/ceph/ceph-client.admin.28011.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28011.log.gz 2024-04-23T02:00:21.927 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56241.log.gz 2024-04-23T02:00:21.927 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28698.log 2024-04-23T02:00:21.927 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36852.log 2024-04-23T02:00:21.927 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37438.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37438.log.gz 2024-04-23T02:00:21.928 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.28698.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.28698.log.gz 2024-04-23T02:00:21.928 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53916.log 2024-04-23T02:00:21.928 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46144.log 2024-04-23T02:00:21.928 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.53916.log: /var/log/ceph/ceph-client.admin.36852.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.36852.log.gz 2024-04-23T02:00:21.928 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.53916.log.gz 2024-04-23T02:00:21.928 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65918.log 2024-04-23T02:00:21.928 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50955.log 2024-04-23T02:00:21.928 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.46144.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46144.log.gz 2024-04-23T02:00:21.928 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.65918.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.65918.log.gz 2024-04-23T02:00:21.928 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62870.log 2024-04-23T02:00:21.928 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61414.log 2024-04-23T02:00:21.928 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.50955.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50955.log.gz 2024-04-23T02:00:21.928 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.62870.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.62870.log.gz 2024-04-23T02:00:21.928 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61235.log 2024-04-23T02:00:21.929 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49144.log 2024-04-23T02:00:21.929 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.61414.log: 48.0% -- replaced with /var/log/ceph/ceph-client.admin.61414.log.gz 2024-04-23T02:00:21.929 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.61235.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.61235.log.gz 2024-04-23T02:00:21.929 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51047.log 2024-04-23T02:00:21.929 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62033.log 2024-04-23T02:00:21.929 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.49144.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49144.log.gz 2024-04-23T02:00:21.929 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.51047.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51047.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32896.log 2024-04-23T02:00:21.929 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-04-23T02:00:21.929 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48488.log 2024-04-23T02:00:21.929 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.62033.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.62033.log.gz 2024-04-23T02:00:21.929 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.32896.log: 50.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.32896.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.29263.log 2024-04-23T02:00:21.929 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-04-23T02:00:21.929 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53747.log 2024-04-23T02:00:21.929 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.48488.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48488.log.gz 2024-04-23T02:00:21.929 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.29263.log: 49.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58122.log 2024-04-23T02:00:21.930 INFO:teuthology.orchestra.run.smithi090.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29263.log.gz 2024-04-23T02:00:21.930 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64615.log 2024-04-23T02:00:21.930 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.53747.log: /var/log/ceph/ceph-client.admin.58122.log: 50.0% 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53747.log.gz 2024-04-23T02:00:21.930 INFO:teuthology.orchestra.run.smithi090.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58122.log.gz 2024-04-23T02:00:21.930 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53469.log 2024-04-23T02:00:21.930 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.64615.log: -- /var/log/ceph/ceph-client.admin.65482.log 2024-04-23T02:00:21.930 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.64615.log.gz 2024-04-23T02:00:21.930 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.53469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39600.log 2024-04-23T02:00:21.930 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.65482.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53469.log.gz 2024-04-23T02:00:21.930 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.65482.log.gz 2024-04-23T02:00:21.930 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44972.log 2024-04-23T02:00:21.930 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.39600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48886.log 2024-04-23T02:00:21.930 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39600.log.gz 2024-04-23T02:00:21.930 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.44972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45806.log 2024-04-23T02:00:21.931 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.48886.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44972.log.gz 2024-04-23T02:00:21.931 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48886.log.gz 2024-04-23T02:00:21.931 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29346.log 2024-04-23T02:00:21.931 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.45806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49322.log 2024-04-23T02:00:21.931 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45806.log.gz 2024-04-23T02:00:21.931 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.29346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39838.log 2024-04-23T02:00:21.931 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.49322.log: 50.0% 50.4% -- replaced with /var/log/ceph/ceph-client.admin.49322.log.gz 2024-04-23T02:00:21.931 INFO:teuthology.orchestra.run.smithi090.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29346.log.gz 2024-04-23T02:00:21.931 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28263.log 2024-04-23T02:00:21.931 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.39838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39089.log 2024-04-23T02:00:21.932 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39838.log.gz 2024-04-23T02:00:21.932 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.28263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43084.log 2024-04-23T02:00:21.932 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.39089.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28263.log.gz 2024-04-23T02:00:21.932 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39089.log.gz 2024-04-23T02:00:21.932 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43571.log 2024-04-23T02:00:21.932 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.43084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34057.log 2024-04-23T02:00:21.932 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43084.log.gz 2024-04-23T02:00:21.932 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.43571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40259.log 2024-04-23T02:00:21.932 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34057.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43571.log.gz 2024-04-23T02:00:21.932 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34057.log.gz 2024-04-23T02:00:21.932 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49402.log 2024-04-23T02:00:21.932 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.40259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49920.log 2024-04-23T02:00:21.932 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40259.log.gz 2024-04-23T02:00:21.932 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.49402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56032.log 2024-04-23T02:00:21.932 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.49920.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49402.log.gz 2024-04-23T02:00:21.933 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49920.log.gz 2024-04-23T02:00:21.933 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45230.log 2024-04-23T02:00:21.933 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46916.log 2024-04-23T02:00:21.933 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.56032.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56032.log.gz 2024-04-23T02:00:21.933 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.45230.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.45230.log.gz 2024-04-23T02:00:21.933 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58819.log 2024-04-23T02:00:21.933 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31214.log 2024-04-23T02:00:21.933 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.46916.log: /var/log/ceph/ceph-client.admin.58819.log: 50.0% 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46916.log.gz 2024-04-23T02:00:21.933 INFO:teuthology.orchestra.run.smithi090.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58819.log.gz 2024-04-23T02:00:21.933 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28562.log 2024-04-23T02:00:21.933 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52893.log 2024-04-23T02:00:21.933 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.31214.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31214.log.gz 2024-04-23T02:00:21.933 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.28562.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28562.log.gz 2024-04-23T02:00:21.934 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45868.log 2024-04-23T02:00:21.934 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39558.log 2024-04-23T02:00:21.934 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.45868.log: /var/log/ceph/ceph-client.admin.52893.log: 49.6% 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45868.log.gz 2024-04-23T02:00:21.934 INFO:teuthology.orchestra.run.smithi090.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52893.log.gz 2024-04-23T02:00:21.934 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65040.log 2024-04-23T02:00:21.934 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.39558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42516.log 2024-04-23T02:00:21.934 INFO:teuthology.orchestra.run.smithi090.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39558.log.gz 2024-04-23T02:00:21.934 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.65040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57533.log 2024-04-23T02:00:21.934 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.42516.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.65040.log.gz 2024-04-23T02:00:21.934 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42516.log.gz 2024-04-23T02:00:21.934 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37498.log 2024-04-23T02:00:21.934 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.57533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57353.log 2024-04-23T02:00:21.934 INFO:teuthology.orchestra.run.smithi090.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.57533.log.gz 2024-04-23T02:00:21.934 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29396.log 2024-04-23T02:00:21.935 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.57353.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37498.log.gz 2024-04-23T02:00:21.935 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.57353.log.gz 2024-04-23T02:00:21.935 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65264.log 2024-04-23T02:00:21.935 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.29396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40539.log 2024-04-23T02:00:21.935 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29396.log.gz 2024-04-23T02:00:21.935 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.65264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57764.log 2024-04-23T02:00:21.935 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.40539.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.65264.log.gz 2024-04-23T02:00:21.935 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40539.log.gz 2024-04-23T02:00:21.935 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50289.log 2024-04-23T02:00:21.935 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.57764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59037.log 2024-04-23T02:00:21.935 INFO:teuthology.orchestra.run.smithi090.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.57764.log.gz 2024-04-23T02:00:21.935 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.0.50289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55542.log 2024-04-23T02:00:21.935 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.59037.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.59037.log.gz 2024-04-23T02:00:21.935 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63713.log 2024-04-23T02:00:21.936 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.55542.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55542.log.gz 2024-04-23T02:00:21.936 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50257.log 2024-04-23T02:00:21.936 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.63713.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.63713.log.gz 2024-04-23T02:00:21.936 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59138.log 2024-04-23T02:00:21.936 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.50257.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50257.log.gz 2024-04-23T02:00:21.936 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53658.log 2024-04-23T02:00:21.936 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.59138.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.59138.log.gz 2024-04-23T02:00:21.936 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31644.log 2024-04-23T02:00:21.936 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.53658.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53658.log.gz 2024-04-23T02:00:21.936 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43760.log 2024-04-23T02:00:21.936 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.31644.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31644.log.gz 2024-04-23T02:00:21.936 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41175.log 2024-04-23T02:00:21.936 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.43760.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43760.log.gz 2024-04-23T02:00:21.936 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49500.log 2024-04-23T02:00:21.936 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.41175.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41175.log.gz 2024-04-23T02:00:21.937 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57273.log 2024-04-23T02:00:21.937 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.49500.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49500.log.gz 2024-04-23T02:00:21.937 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55932.log 2024-04-23T02:00:21.937 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.57273.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.57273.log.gz 2024-04-23T02:00:21.937 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56866.log 2024-04-23T02:00:21.937 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.55932.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.55932.log.gz 2024-04-23T02:00:21.937 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58211.log 2024-04-23T02:00:21.937 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.56866.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.56866.log.gz 2024-04-23T02:00:21.937 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44325.log 2024-04-23T02:00:21.937 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.58211.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.58211.log.gz 2024-04-23T02:00:21.937 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51723.log 2024-04-23T02:00:21.937 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.44325.log: 50.8% -- replaced with /var/log/ceph/ceph-client.admin.44325.log.gz 2024-04-23T02:00:21.937 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34504.log 2024-04-23T02:00:21.937 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.51723.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51723.log.gz 2024-04-23T02:00:21.937 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47643.log 2024-04-23T02:00:21.938 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34504.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34504.log.gz 2024-04-23T02:00:21.938 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61316.log 2024-04-23T02:00:21.938 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.47643.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47643.log.gz 2024-04-23T02:00:21.938 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63804.log 2024-04-23T02:00:21.938 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.61316.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.61316.log.gz 2024-04-23T02:00:21.938 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33561.log 2024-04-23T02:00:21.938 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.63804.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.63804.log.gz 2024-04-23T02:00:21.938 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59811.log 2024-04-23T02:00:21.938 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33561.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.33561.log.gz 2024-04-23T02:00:21.938 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30688.log 2024-04-23T02:00:21.938 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.59811.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.59811.log.gz 2024-04-23T02:00:21.938 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44414.log 2024-04-23T02:00:21.938 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.30688.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30688.log.gz 2024-04-23T02:00:21.938 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54007.log 2024-04-23T02:00:21.938 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.44414.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44414.log.gz 2024-04-23T02:00:21.939 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33512.log 2024-04-23T02:00:21.939 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.54007.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.54007.log.gz 2024-04-23T02:00:21.939 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30130.log 2024-04-23T02:00:21.939 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33512.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33512.log.gz 2024-04-23T02:00:21.939 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35020.log 2024-04-23T02:00:21.939 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.30130.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.30130.log.gz 2024-04-23T02:00:21.939 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64209.log 2024-04-23T02:00:21.939 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35020.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35020.log.gz 2024-04-23T02:00:21.939 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50033.log 2024-04-23T02:00:21.939 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.64209.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.64209.log.gz 2024-04-23T02:00:21.939 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56499.log 2024-04-23T02:00:21.939 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.50033.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50033.log.gz 2024-04-23T02:00:21.964 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41475.log 2024-04-23T02:00:21.965 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.56499.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56499.log.gz 2024-04-23T02:00:21.965 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32976.log 2024-04-23T02:00:21.965 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.41475.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41475.log.gz 2024-04-23T02:00:21.965 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27901.log 2024-04-23T02:00:21.965 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.32976.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32976.log.gz 2024-04-23T02:00:21.965 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51270.log 2024-04-23T02:00:21.965 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.27901.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.27901.log.gz 2024-04-23T02:00:21.965 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30648.log 2024-04-23T02:00:21.965 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.51270.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51270.log.gz 2024-04-23T02:00:21.965 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65083.log 2024-04-23T02:00:21.965 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.30648.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.30648.log.gz 2024-04-23T02:00:21.966 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49731.log 2024-04-23T02:00:21.966 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.65083.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.65083.log.gz 2024-04-23T02:00:21.966 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54840.log 2024-04-23T02:00:21.966 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.49731.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.49731.log.gz 2024-04-23T02:00:21.966 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29480.log 2024-04-23T02:00:21.966 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.54840.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.54840.log.gz 2024-04-23T02:00:21.966 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33690.log 2024-04-23T02:00:21.966 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.29480.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.29480.log.gz 2024-04-23T02:00:21.966 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57442.log 2024-04-23T02:00:21.966 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33690.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33690.log.gz 2024-04-23T02:00:21.966 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45428.log 2024-04-23T02:00:21.966 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.57442.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.57442.log.gz 2024-04-23T02:00:21.966 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59344.log 2024-04-23T02:00:21.966 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.45428.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45428.log.gz 2024-04-23T02:00:21.967 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37333.log 2024-04-23T02:00:21.967 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.59344.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.59344.log.gz 2024-04-23T02:00:21.967 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40399.log 2024-04-23T02:00:21.967 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37333.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37333.log.gz 2024-04-23T02:00:21.967 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62666.log 2024-04-23T02:00:21.967 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.40399.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40399.log.gz 2024-04-23T02:00:21.967 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56081.log 2024-04-23T02:00:21.967 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.62666.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.62666.log.gz 2024-04-23T02:00:21.967 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32667.log 2024-04-23T02:00:21.967 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.56081.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56081.log.gz 2024-04-23T02:00:21.967 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64957.log 2024-04-23T02:00:21.967 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.32667.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32667.log.gz 2024-04-23T02:00:21.967 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31039.log 2024-04-23T02:00:21.968 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.64957.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.64957.log.gz 2024-04-23T02:00:21.968 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54503.log 2024-04-23T02:00:21.968 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.31039.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31039.log.gz 2024-04-23T02:00:21.968 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57135.log 2024-04-23T02:00:21.968 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.54503.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.54503.log.gz 2024-04-23T02:00:21.968 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29096.log 2024-04-23T02:00:21.968 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.57135.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.57135.log.gz 2024-04-23T02:00:21.968 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54939.log 2024-04-23T02:00:21.968 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.29096.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.29096.log.gz 2024-04-23T02:00:21.968 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37594.log 2024-04-23T02:00:21.968 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.54939.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54939.log.gz 2024-04-23T02:00:21.968 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38244.log 2024-04-23T02:00:21.968 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37594.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37594.log.gz 2024-04-23T02:00:21.969 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35725.log 2024-04-23T02:00:21.969 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38244.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38244.log.gz 2024-04-23T02:00:21.969 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53378.log 2024-04-23T02:00:21.969 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35725.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35725.log.gz 2024-04-23T02:00:21.969 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66196.log 2024-04-23T02:00:21.969 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.53378.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53378.log.gz 2024-04-23T02:00:21.969 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58899.log 2024-04-23T02:00:21.969 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.66196.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.66196.log.gz 2024-04-23T02:00:21.969 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38920.log 2024-04-23T02:00:21.969 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.58899.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.58899.log.gz 2024-04-23T02:00:21.969 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38200.log 2024-04-23T02:00:21.969 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38920.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38920.log.gz 2024-04-23T02:00:21.969 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54245.log 2024-04-23T02:00:21.970 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38200.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38200.log.gz 2024-04-23T02:00:21.970 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48299.log 2024-04-23T02:00:21.970 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.54245.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.54245.log.gz 2024-04-23T02:00:21.970 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54810.log 2024-04-23T02:00:21.970 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.48299.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48299.log.gz 2024-04-23T02:00:21.970 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38473.log 2024-04-23T02:00:21.970 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.54810.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54810.log.gz 2024-04-23T02:00:21.970 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64779.log 2024-04-23T02:00:21.970 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38473.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38473.log.gz 2024-04-23T02:00:21.970 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36953.log 2024-04-23T02:00:21.970 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.64779.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.64779.log.gz 2024-04-23T02:00:21.970 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46589.log 2024-04-23T02:00:21.970 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.36953.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.36953.log.gz 2024-04-23T02:00:21.971 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58779.log 2024-04-23T02:00:21.971 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.46589.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46589.log.gz 2024-04-23T02:00:21.971 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39129.log 2024-04-23T02:00:21.971 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.58779.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.58779.log.gz 2024-04-23T02:00:21.971 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32269.log 2024-04-23T02:00:21.971 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.39129.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39129.log.gz 2024-04-23T02:00:21.971 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52199.log 2024-04-23T02:00:21.971 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.32269.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32269.log.gz 2024-04-23T02:00:21.971 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62708.log 2024-04-23T02:00:21.971 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.52199.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.52199.log.gz 2024-04-23T02:00:21.971 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65440.log 2024-04-23T02:00:21.971 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.62708.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.62708.log.gz 2024-04-23T02:00:21.971 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44745.log 2024-04-23T02:00:21.971 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.65440.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.65440.log.gz 2024-04-23T02:00:21.972 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60001.log 2024-04-23T02:00:21.972 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.44745.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44745.log.gz 2024-04-23T02:00:21.972 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26993.log 2024-04-23T02:00:21.972 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.60001.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.60001.log.gz 2024-04-23T02:00:21.972 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63357.log 2024-04-23T02:00:21.972 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.26993.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.26993.log.gz 2024-04-23T02:00:21.972 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41878.log 2024-04-23T02:00:21.972 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.63357.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.63357.log.gz 2024-04-23T02:00:21.972 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59097.log 2024-04-23T02:00:21.972 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.41878.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41878.log.gz 2024-04-23T02:00:21.972 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29165.log 2024-04-23T02:00:21.972 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.59097.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.59097.log.gz 2024-04-23T02:00:21.973 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60749.log 2024-04-23T02:00:21.973 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.29165.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29165.log.gz 2024-04-23T02:00:21.973 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45847.log 2024-04-23T02:00:21.973 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.60749.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.60749.log.gz 2024-04-23T02:00:21.973 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51683.log 2024-04-23T02:00:21.973 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.45847.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.45847.log.gz 2024-04-23T02:00:21.973 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44563.log 2024-04-23T02:00:21.981 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.51683.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.51683.log.gz 2024-04-23T02:00:21.981 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64859.log 2024-04-23T02:00:21.982 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.44563.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44563.log.gz 2024-04-23T02:00:21.982 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65776.log 2024-04-23T02:00:21.982 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.64859.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.64859.log.gz 2024-04-23T02:00:21.982 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52853.log 2024-04-23T02:00:21.982 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.65776.log: 48.0% -- replaced with /var/log/ceph/ceph-client.admin.65776.log.gz 2024-04-23T02:00:21.982 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46103.log 2024-04-23T02:00:21.982 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.52853.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.52853.log.gz 2024-04-23T02:00:21.982 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60609.log 2024-04-23T02:00:21.982 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.46103.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46103.log.gz 2024-04-23T02:00:21.982 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60050.log 2024-04-23T02:00:21.982 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.60609.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.60609.log.gz 2024-04-23T02:00:21.982 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48170.log 2024-04-23T02:00:21.983 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.60050.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.60050.log.gz 2024-04-23T02:00:21.983 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61365.log 2024-04-23T02:00:21.983 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.48170.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48170.log.gz 2024-04-23T02:00:21.983 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33016.log 2024-04-23T02:00:21.983 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.61365.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.61365.log.gz 2024-04-23T02:00:21.983 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66406.log 2024-04-23T02:00:21.983 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33016.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33016.log.gz 2024-04-23T02:00:21.983 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51339.log 2024-04-23T02:00:21.983 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.66406.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.66406.log.gz 2024-04-23T02:00:21.983 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38284.log 2024-04-23T02:00:21.983 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.51339.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51339.log.gz 2024-04-23T02:00:21.983 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35149.log 2024-04-23T02:00:21.983 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38284.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38284.log.gz 2024-04-23T02:00:21.983 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30550.log 2024-04-23T02:00:21.983 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35149.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35149.log.gz 2024-04-23T02:00:21.984 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65920.log 2024-04-23T02:00:21.984 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.30550.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.30550.log.gz 2024-04-23T02:00:21.984 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28818.log 2024-04-23T02:00:21.984 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.65920.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.65920.log.gz 2024-04-23T02:00:21.984 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64042.log 2024-04-23T02:00:21.984 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.28818.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28818.log.gz 2024-04-23T02:00:21.984 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67162.log 2024-04-23T02:00:21.984 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.64042.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.64042.log.gz 2024-04-23T02:00:21.984 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54552.log 2024-04-23T02:00:21.984 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.67162.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.67162.log.gz 2024-04-23T02:00:21.984 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41035.log 2024-04-23T02:00:21.984 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.54552.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54552.log.gz 2024-04-23T02:00:21.984 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55983.log 2024-04-23T02:00:21.987 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.41035.log: 47.6% -- replaced with /var/log/ceph/ceph-client.admin.41035.log.gz 2024-04-23T02:00:21.987 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42236.log 2024-04-23T02:00:21.987 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.55983.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55983.log.gz 2024-04-23T02:00:21.987 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50131.log 2024-04-23T02:00:21.987 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.42236.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42236.log.gz 2024-04-23T02:00:21.987 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36614.log 2024-04-23T02:00:21.987 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.50131.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50131.log.gz 2024-04-23T02:00:21.987 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29214.log 2024-04-23T02:00:21.988 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.36614.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.36614.log.gz 2024-04-23T02:00:21.988 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.29214.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29214.log.gz 2024-04-23T02:00:22.005 INFO:teuthology.orchestra.run.smithi090.stderr: 89.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-23T02:00:22.016 INFO:teuthology.orchestra.run.smithi090.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.0.33413.log.gz 2024-04-23T02:00:22.193 INFO:teuthology.orchestra.run.smithi090.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.0.41641.log.gz 2024-04-23T02:00:22.258 INFO:teuthology.orchestra.run.smithi090.stderr: 93.9% -- replaced with /var/log/ceph/ceph-client.0.37828.log.gz 2024-04-23T02:00:22.375 INFO:teuthology.orchestra.run.smithi090.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.0.50289.log.gz 2024-04-23T02:00:42.124 INFO:teuthology.orchestra.run.smithi113.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-23T02:00:42.620 INFO:teuthology.orchestra.run.smithi113.stderr: 92.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-23T02:00:42.622 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-04-23T02:00:42.622 INFO:teuthology.orchestra.run.smithi113.stderr:real 0m21.423s 2024-04-23T02:00:42.622 INFO:teuthology.orchestra.run.smithi113.stderr:user 0m40.894s 2024-04-23T02:00:42.622 INFO:teuthology.orchestra.run.smithi113.stderr:sys 0m1.494s 2024-04-23T02:00:43.346 INFO:teuthology.orchestra.run.smithi090.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-23T02:00:45.573 INFO:teuthology.orchestra.run.smithi090.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-23T02:00:45.575 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-04-23T02:00:45.575 INFO:teuthology.orchestra.run.smithi090.stderr:real 0m24.415s 2024-04-23T02:00:45.575 INFO:teuthology.orchestra.run.smithi090.stderr:user 0m49.692s 2024-04-23T02:00:45.575 INFO:teuthology.orchestra.run.smithi090.stderr:sys 0m2.057s 2024-04-23T02:00:45.575 INFO:tasks.ceph:Archiving logs... 2024-04-23T02:00:45.575 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/log/ceph to /home/teuthworker/archive/yingxin-2024-04-23_01:23:55-crimson-rados-ci-yingxin-crimson-fix-tri-mutex-4-distro-default-smithi/7669459/remote/smithi090/log 2024-04-23T02:00:45.576 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T02:00:54.303 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/log/ceph to /home/teuthworker/archive/yingxin-2024-04-23_01:23:55-crimson-rados-ci-yingxin-crimson-fix-tri-mutex-4-distro-default-smithi/7669459/remote/smithi113/log 2024-04-23T02:00:54.304 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T02:01:01.404 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-23T02:01:01.414 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-23T02:01:01.414 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T02:01:01.417 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T02:01:01.509 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-23T02:01:01.547 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-23T02:01:01.547 DEBUG:teuthology.orchestra.run.smithi090:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T02:01:01.551 DEBUG:teuthology.orchestra.run.smithi113:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T02:01:01.564 INFO:teuthology.orchestra.run.smithi090.stderr:bash: line 1: ntpq: command not found 2024-04-23T02:01:01.569 INFO:teuthology.orchestra.run.smithi090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T02:01:01.569 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================== 2024-04-23T02:01:01.569 INFO:teuthology.orchestra.run.smithi090.stdout:^- hv01.front.sepia.ceph.com 2 6 377 38 +980us[ +946us] +/- 590ms 2024-04-23T02:01:01.569 INFO:teuthology.orchestra.run.smithi090.stdout:^* hv02.front.sepia.ceph.com 3 6 377 33 -739us[ -773us] +/- 63ms 2024-04-23T02:01:01.570 INFO:teuthology.orchestra.run.smithi090.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 +928us[ +894us] +/- 73ms 2024-04-23T02:01:01.570 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T02:01:01.571 INFO:teuthology.orchestra.run.smithi113.stderr:bash: line 1: ntpq: command not found 2024-04-23T02:01:01.573 INFO:teuthology.orchestra.run.smithi113.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T02:01:01.573 INFO:teuthology.orchestra.run.smithi113.stdout:=============================================================================== 2024-04-23T02:01:01.574 INFO:teuthology.orchestra.run.smithi113.stdout:^- hv01.front.sepia.ceph.com 2 6 377 35 +970us[ +970us] +/- 590ms 2024-04-23T02:01:01.574 INFO:teuthology.orchestra.run.smithi113.stdout:^* hv02.front.sepia.ceph.com 3 6 377 37 -756us[ -767us] +/- 63ms 2024-04-23T02:01:01.574 INFO:teuthology.orchestra.run.smithi113.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 +925us[ +925us] +/- 73ms 2024-04-23T02:01:01.574 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T02:01:01.576 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-23T02:01:01.592 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-23T02:01:01.593 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-23T02:01:01.622 DEBUG:teuthology.orchestra.run.smithi090:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T02:01:01.706 DEBUG:teuthology.orchestra.run.smithi113:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T02:01:01.769 DEBUG:teuthology.orchestra.run.smithi090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T02:01:01.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T02:01:01.798 DEBUG:teuthology.orchestra.run.smithi113:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T02:01:01.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T02:01:01.827 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-23T02:01:01.842 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-23T02:01:01.880 INFO:teuthology.task.internal:Duration was 1414.871775 seconds 2024-04-23T02:01:01.880 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-23T02:01:01.916 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-23T02:01:01.916 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T02:01:01.919 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T02:01:01.953 INFO:teuthology.orchestra.run.smithi090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T02:01:01.956 INFO:teuthology.orchestra.run.smithi113.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T02:01:02.274 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-23T02:01:02.274 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi090.front.sepia.ceph.com 2024-04-23T02:01:02.274 DEBUG:teuthology.orchestra.run.smithi090:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T02:01:02.335 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi113.front.sepia.ceph.com 2024-04-23T02:01:02.335 DEBUG:teuthology.orchestra.run.smithi113:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T02:01:02.359 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-23T02:01:02.359 DEBUG:teuthology.orchestra.run.smithi090:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T02:01:02.378 DEBUG:teuthology.orchestra.run.smithi113:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T02:01:02.424 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-23T02:01:02.424 DEBUG:teuthology.orchestra.run.smithi090:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T02:01:02.443 DEBUG:teuthology.orchestra.run.smithi113:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T02:01:02.569 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-23T02:01:02.581 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-23T02:01:02.581 DEBUG:teuthology.orchestra.run.smithi090:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T02:01:02.605 DEBUG:teuthology.orchestra.run.smithi113:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T02:01:02.629 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-23T02:01:02.639 DEBUG:teuthology.orchestra.run.smithi090:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T02:01:02.648 DEBUG:teuthology.orchestra.run.smithi113:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T02:01:02.667 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = core 2024-04-23T02:01:02.691 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern = core 2024-04-23T02:01:02.711 DEBUG:teuthology.orchestra.run.smithi090:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T02:01:02.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T02:01:02.744 DEBUG:teuthology.orchestra.run.smithi113:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T02:01:02.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T02:01:02.766 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-23T02:01:02.776 INFO:teuthology.task.internal:Transferring archived files... 2024-04-23T02:01:02.777 DEBUG:teuthology.misc:Transferring archived files from smithi090:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yingxin-2024-04-23_01:23:55-crimson-rados-ci-yingxin-crimson-fix-tri-mutex-4-distro-default-smithi/7669459/remote/smithi090 2024-04-23T02:01:02.777 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T02:01:02.821 DEBUG:teuthology.misc:Transferring archived files from smithi113:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yingxin-2024-04-23_01:23:55-crimson-rados-ci-yingxin-crimson-fix-tri-mutex-4-distro-default-smithi/7669459/remote/smithi113 2024-04-23T02:01:02.821 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T02:01:02.856 INFO:teuthology.task.internal:Removing archive directory... 2024-04-23T02:01:02.857 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T02:01:02.860 DEBUG:teuthology.orchestra.run.smithi113:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T02:01:02.905 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-23T02:01:02.917 INFO:teuthology.task.internal:Not uploading archives. 2024-04-23T02:01:02.917 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-23T02:01:02.943 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-23T02:01:02.944 DEBUG:teuthology.orchestra.run.smithi090:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T02:01:02.947 DEBUG:teuthology.orchestra.run.smithi113:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T02:01:02.958 INFO:teuthology.orchestra.run.smithi090.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 23 02:01 /home/ubuntu/cephtest 2024-04-23T02:01:02.961 INFO:teuthology.orchestra.run.smithi113.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 23 02:01 /home/ubuntu/cephtest 2024-04-23T02:01:02.962 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-23T02:01:02.972 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-23T02:01:03.044 INFO:teuthology.run:Summary data: description: crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/simple thrashosds-health workloads/radosbench-high-concurrency} duration: 1414.8717753887177 flavor: crimson owner: scheduled_yingxin@teuthology success: true 2024-04-23T02:01:03.044 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-23T02:01:03.145 INFO:teuthology.run:pass